UMC memory pointer system revisited

This is the place for queries that don't fit in any of the other categories.

UMC memory pointer system revisited

Postby Tcll » Mon Mar 24, 2014 5:00 pm

so I was pretty much finished with the data types when I got hit with structs which made me re-look at my entire design...
it's possible yes, but now I'm entirely reconsidering my behavioral patterns to match that in the C compiler... (including the data conversions)

so to start off...
if I was to reference a bu32 type as a pointer to a s16 type,
am I right in converting the original bu32 type to s16, or would it still be bu32??

just to point out, yes this is still python.
the pointer system is being used to track your position in a file which in certain cases can also be called a memory space.

no this doesn't work with a file directly.
the "file" is a 'B' array

the whole pointer system is more of a library that operates off of this virtual memory space with special functions to allocate space before writing to it,
or to pre-read data for verification.

I just need to know how the compiler would manage such issues if they were to occure...
(there may be some systems that actually abuse this behavior)

OT: also... I don't want this thread to end anytime soon, so excuse me before-hand for necro-posting in the future. ;)
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Tue Mar 25, 2014 1:44 am

well I think here mostly verified my Q:
http://forums.devshed.com/programming-4 ... 60334.html
no, it doesn't change the original data type, only the value.

I'm gonna have to play around with multi-threading to sync the data-class to the value when it changes >.>

but if anyone else has anything different to say, please let me know. :)
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby stranac » Tue Mar 25, 2014 10:42 am

Tcll wrote:but if anyone else has anything different to say, please let me know.

I still think what you're doing is stupid, and there is probably a better way to do whatever it is you want to do.
As it stands, you're trying to make python into C with more awkward syntax and no benefits...
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1209
Joined: Thu Feb 07, 2013 3:42 pm

Re: UMC memory pointer system revisited

Postby Tcll » Thu Mar 27, 2014 2:40 am

stranac wrote:and there is probably a better way to do whatever it is you want to do.

design a very simple interface to read a file as virtual memory with pointer operations and C-like behavior.
that's what I'm trying to do.

however I'm supporting extended operations for future development such as bf(2) which can also be called bf16

the benefit of all this, where to start... lol
how bout a very simple interface to manage file data.

basically, compare my system to Blender...
you need at most 80% less code to do the same thing that you would with a Blender script.

unlike blender, which makes things more complex from 24x to 26x,
UMC makes things simpler and automates with the option to specify otherwize.


so why the C-like behavior?
most of the files I deal with are binary files based on RAM structures from other systems.
(systems of which are all based on C)

but I'm not only supporting binary, I'm also building systems for advanced text services for text-based files.

later on I intend to turn this into a sort-of extendable logic-based compiler for various languages.
meaning this thing will be able to convert Python into Visual Basic w/o the use of the CLI.
but that's something I intend to build quite a few devs later... heh
(when I change the name from UMC to UGC)


as for currently, I do need a way to simulate C-type structs using this:

name1 = struct()
name2 = name1()

type(name2) == name1
type(name1) == struct

any ideas how I could do that??
note: this has nothing to do with the struct module.


EDIT:
take into account that I'm extremely horrible at wording myself >.<
it's a symptom of my autism... sorry :(
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Thu Mar 27, 2014 4:17 am

well here's a little somthing to start off with:
Code: Select all
import FILE, DATA

bf32 = DATA.bf32 #this will be taken care of by the outer interface later

class _SUB_STRUCT(object):
    def __init__(this): pass
           
class struct(object):
    def __init__(this, size, **names):
        #print size, kwargs

        this.SS = _SUB_STRUCT()
        this.SS.__dict__['_size'] = size
        this.SS.__dict__.update(names)

    def __call__(this): return this.SS


#to be written in scripts:

x = struct( 12,
           x = bf32,
           y = bf32,
           z = bf32
           )

d = x()


print 'test 1:', ['failed','passed'][type(x) == struct], '( type(x) == struct )'
print 'test 2:', ['failed','passed'][type(d) == x], '( type(d) == x )'
print 'test 3:', ['failed','passed'][type(d.x) == bf32], '( type(d.x) == bf32 ) #not yet, bf32 isn't told to read just yet.'


results:
Code: Select all
>>>
test 1: passed ( type(x) == struct )
test 2: failed ( type(d) == x )
test 3: failed ( type(d.x) == bf32 ) #not yet, bf32 isn't told to read just yet.
>>>
as expected
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Thu Mar 27, 2014 12:48 pm

well, I've gotten it somewhat working like this:
code:
Code: Select all
import FILE, DATA

bf32 = DATA.bf32

class _STRUCT(object):
    def __init__(this): pass
    def __call__(this, size, **names):
        #print size, names

        class _SUB_STRUCT(object):
            _names = names
            _size = size
            def __init__(this, *values):
                this.__dict__.update(this._names)

        return _SUB_STRUCT

struct = _STRUCT()

structType = type

x = struct( 12,
           x = bf32,
           y = bf32,
           z = bf32
           )

d = x()


print 'test 1:', ['failed','passed'][type(x) == structType], '( type(x) == structType )'
print 'test 2:', ['failed','passed'][type(d) == x], '( type(d) == x )'
print 'test 3:', ['failed','passed'][type(d.x) == bf32], '( type(d.x) == bf32 ) #not yet'

results:
Code: Select all
>>>
test 1: passed ( type(x) == structType )
test 2: passed ( type(d) == x )
test 3: failed ( type(d.x) == bf32 ) #not yet
>>>


I wanna try to get it to recognize 'struct' instead of 'structType' >.<
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Thu Mar 27, 2014 3:05 pm

hmm...
anyone know of a hack to get the original order of **kwargs??

otherwize, I'll have to make a few changes to:

struct( 12,
x = bf32,
y = bf32,
z = bf32
)

such as:

x = struct( 12,
{'x' : bf32},
{'y' : bf32},
{'z' : bf32},
)

or even:

struct( 12, 'x,y,z',
x = bf32,
z = bf32,
y = bf32,
)
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Thu Mar 27, 2014 10:47 pm

alright, so here we are...
here's the script:
Code: Select all
import FILE
from FILE import *
bu32,bu16,bf32 = DATA.bu32,DATA.bu16,DATA.bf32
struct = DATA.struct; structType = DATA.structType

# --- UMC-script operations:

ImportFile('FitPikachu00.mdl0')

header = struct( 32, #allocation size
    'magic,datasize,dataoffset', #variable order (to be cleaned up later)
   
    magic = bu32, #string(4),
    datasize = bu32,
    dataoffset = bu32
   
    #the rest is automatically skipped

    )() #read

# --- Tests from previous script:

x = struct( 12, 'x,y,z',
    x = bf32,
    y = bf32,
    z = bf32
)

d = x()
d2 = x()

#---

print
print 'test 1:', ['failed','passed'][type(x) == structType], '( type(x) == structType )'
print 'test 2:', ['failed','passed'][type(d) == x], '( type(d) == x )'
print 'test 3:', ['failed','passed'][type(d.x) == bf32], '( type(d.x) == bf32 )'
print 'test 4:', ['failed','passed'][type(d) == type(d2)], '( type(d) == type(d2) )\n\n'

mptr = ref(header.magic)

print 'header.magic == "MDL0":', header.magic == "MDL0", header.magic

v = 0
for c in 'TEX0': v = (v<<8)|ord(c)

print '\noriginal:',header.magic,header.magic.__class__,'\nfile[0:4]: "',
for i in FILE._current.data[0:4]: print chr(i),
print '"\n'

deref(mptr, DATA.bu32).set(v)

print 'replaced:',header.magic,header.magic.__class__,'\nfile[0:4]: "',
for i in FILE._current.data[0:4]: print chr(i),
print '"\n'

deref(mptr, DATA.bu16).set(v)

print 'modified:',header.magic,header.magic.__class__,'\nfile[0:4]: "',
for i in FILE._current.data[0:4]: print chr(i),
print '"\n'

#print 'result:\nfile[0:4]: "',
#for i in FILE._current.data[0:4]: print chr(i),
#print '"\n'

# ---

raw_input() #wait for activity


here's the data parsed throughout this test:
Code: Select all
4D 44 4C 30 00 02 A1 60 00 00 00 09 FF FF FF 80
00 00 05 18 00 00 05 60 00 00 08 78 00 00 08 E0
00 00 09 48 00 00 09 70 00 00 09 D8 00 00 0A 90
00 00 0B 48 00 00 0B B0


and here's the results:
Code: Select all
>>>
{'dataoffset': <class 'DATA._u.bu_4'>, 'datasize': <class 'DATA._u.bu_4'>, 'magic': <class 'DATA._u.bu_4'>}
{'y': <class 'DATA._f.bf_4'>, 'x': <class 'DATA._f.bf_4'>, 'z': <class 'DATA._f.bf_4'>}
{'y': <class 'DATA._f.bf_4'>, 'x': <class 'DATA._f.bf_4'>, 'z': <class 'DATA._f.bf_4'>}

test 1: passed ( type(x) == structType )
test 2: passed ( type(d) == x )
test 3: passed ( type(d.x) == bf32 )
test 4: passed ( type(d) == type(d2) )


header.magic == "MDL0": False 1296321584

original: 1296321584 <class 'DATA._u.bu_4'>
file[0:4]: " M D L 0 "

replaced: 5567658808379089248 <class 'DATA._u.bu_4'>
file[0:4]: " T E X 0 "

modified: 23912912497274519091904774146 <class 'DATA._u.bu_4'>
file[0:4]: " X 0 X 0 "


>>> d.x
3.32948515124e-42
>>> FILE._current.data[d.x._addr:d.x._addr+4]
array('B', [0, 0, 9, 72])
>>> FILE._current.data[d2.z._addr:d2.z._addr+4]
array('B', [0, 0, 11, 176])
>>>


now to see if I can get the pointer to the struct :3
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Thu May 22, 2014 5:54 am

bump

hey I think I have an idea on how to get rid of the order requirement! =D
I'm not really sure about it, but this article on the inspect module gives me some hope. :)
http://blog.lerner.co.il/making-init-me ... -autoinit/

hopefully I can get things working as expected with this :)
(catching the frame of the referenced UMC type and deriving the order from the variable name BEFORE it gets stored in the dict)

I've already gotten this working somewhat around a different area for UMC_SIDE,
for function referencing to get the line number and column for highlighting the "selected" function in the script based on the selected data in the viewers.
(only the basic script works right now (using a decorator modded when loading UMC_SIDE for UMC's functions)... nothing is actually implemented yet)

again... I only have some hope of getting struct() to work similarly to C, as well as getting an accurate type-reference.
(I don't like using "type(struct()) == structType", I want it to be "type(struct()) == struct")
^ keep in mind: x = struct(); v = x(); type(v) == x; type(x) == structType # (this is already working perfectly)

the problem I'm having, and why I need structType:
print type(struct()) == type >>> True
^struct() returns a class definition (not an instance like struct()())

I know that's probably been known already, but blah. :P
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Mekire » Thu May 22, 2014 6:12 am

It's been said before, but the reason you don't get much help here is because what you are doing is always about as far from anything that should be done in Python as possible. No one wants to work with you to figure out why you are trying to butcher Python so that it acts like C.

As Stranac said:
Stranac wrote:[T]here is probably a better way to do whatever it is you want to do.
As it stands, you're trying to make python into C with more awkward syntax and no benefits...

To be brutally honest, you aren't going to find help here. Personally I just tend to ignore your threads now, because I know what they are going to contain.

-Mek
User avatar
Mekire
 
Posts: 1012
Joined: Thu Feb 07, 2013 11:33 pm
Location: Amakusa, Japan

Re: UMC memory pointer system revisited

Postby Tcll » Thu May 22, 2014 12:53 pm

Mekire wrote:It's been said before, but the reason you don't get much help here is because what you are doing is always about as far from anything that should be done in Python as possible. No one wants to work with you to figure out why you are trying to butcher Python so that it acts like C.

As Stranac said:
Stranac wrote:[T]here is probably a better way to do whatever it is you want to do.
As it stands, you're trying to make python into C with more awkward syntax and no benefits...

To be brutally honest, you aren't going to find help here. Personally I just tend to ignore your threads now, because I know what they are going to contain.

-Mek

ok... PlPcNr.dat is essentially RAM data.

it's because of files like these that I'm trying to build a structuring system similar to C,
since the base scripting that actually uses these files is also a variant of C.

again, I want it to be python, but I want to provide a syntax for my program that makes things easier to operate on both binary and text files.
(python's struct is a horrible and very limited module)

EDIT:
Stranac wrote:[T]here is probably a better way to do whatever it is you want to do.
As it stands, you're trying to make python into C with more awkward syntax and no benefits...

I still son't understand what stranac means by "no benefits"...
if being far simpler to use than current methods isn't a benefit, then what is??

EDIT2:
keep in mind, this system is meant for noobs for taking file formats off the internet and building very simple scripts that work with them.
(python is complex compaired to my system)

EDIT3:
would you like me to write out a small example script that reads the header of a file??
(just so you can see what exactly I'm going for, and where exactly I need the help)
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Kebap » Thu May 22, 2014 1:15 pm

Tcll wrote:would you like me to write out a small example script that reads the header of a file??
(just so you can see what exactly I'm going for, and where exactly I need the help)

Please do show examples and elaborate, what you are going for in the end.

It seems like something huge and complex and never-been-done-before, but if it can indeed help everybody to ease their work, then more power to it!

Also check out the link in my signature, for example:

Question:
How can I use X to do Y?

Answer:
If what you want is to do Y, you should ask that question without pre-supposing the use of a method that may not be appropriate. Questions of this form often indicate a person who is not merely ignorant about X, but confused about what problem Y they are solving and too fixated on the details of their particular situation. It is generally best to ignore such people until they define their problem better.
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 397
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe

Re: UMC memory pointer system revisited

Postby Tcll » Thu May 22, 2014 2:34 pm

yea, I know the thing about asking proper questions and what not...
I don't think properly, so the proper question I want to ask usually comes 2 days after my original post at least.
(sorry I'm a disabled wreck when it comes to socializing) heh

anyways, here's a script using the new system for the MDL0 file format:

Code: Select all
def ImportModel( FileType, Commands ):

    #global header:
    GHeader = struct( 16, # order not used here (this is what I'm going for, but right now would break the script)
        magic = string(4), # "MDL0"
        DataSize = bu32,
        version = bu32,
        BRRESPos = bs32 # this file's position in the outer archive
    )()

    sections = struct( -1, # struct size depends on used fields
        # order can still be used for this reason:
        '''
        Definitions,
        Bones,
        Vertices,
        Normals,
        Colors,
        UVs,'''+\
        ('''
        FurVectors,
        FurLayers,
        ''' if GHeader.version>9 else '')+'''
        Materials,
        TEVs,
        Objects,
        Textures,
        Pallets'''+\
        (''',
        Unk
        ''' if GHeader.version>10 else ''),

        Definitions = bu32,
        Bones = bu32,
        Vertices = bu32,
        Normals  = bu32,
        Colors   = bu32,
        UVs      = bu32,
        FurVectors = bu32,
        FurLayers  = bu32,
        Materials= bu32,
        TEVs = bu32,
        Objects  = bu32,
        Textures = bu32,
        Pallets  = bu32,
        Unk     = bu32,
    ) # note we're not calling a read here

    bounds3 = struct( 24,
        x = bf32, y = bf32, z = bf32,
        X = bf32, Y = bf32, Z = bf32
    )

    header = struct( -1,
        DataSections = sections
        MDL0Name = string( bu8, addr=bu32 )
        HeaderLength = bu32,
        MDL0Offset = bs32,
        unk1 = bu32,
        unk2 = bu32,
        vertCount = bu32,
        faceCount = bu32,
        unk3 = bu32,
        DefCount = bu32,
        unk_flags = bu32,
        unk5 = bu16,
        DataOffset = bu16,
        Bounds = bounds3
    )()

^ note how it looks grouped easier (same effect as using a class, but operates just like a UMC data type).

again, this is made simple for noob programmers.
you can still use whatever you think is best

here's an older code from my last build:
Code: Select all
def ImportModel(FT,Cmd):

    def Offset_Name(offset):
        p=jump(offset-1,label=' -- String_Data')
        Str=string(bu8( label=' -- String_Length'))
        jump(p); return Str

    def Bounds(Type): #MinX,Y(,Z),MaxX,Y(,Z)
        LABEL('\n -- Bounds:')
        return [bf32(),bf32(),bf32(),bf32()]+([bf32(),bf32()] if Type==3 else [])

    #global header:
    magic = string(4, label=' -- MDL0_Magic')
    DataLen = bu32(   label=' -- Data_Length')
    version = bu32(   label=' -- Version')
    brres_hdr = bs32( label=' -- Brres_Header')

    #data sections
    Definits = (bu32(label=' -- Definitions')if version>7 else 0)
    Bones    = (bu32(label=' -- Bones')      if version>7 else 0)
    Vertices = (bu32(label=' -- Vertices')   if version>7 else 0)
    Normals  = (bu32(label=' -- Normals')    if version>7 else 0)
    Colors   = (bu32(label=' -- Colors')     if version>7 else 0)
    UVs      = (bu32(label=' -- UVs')        if version>7 else 0)
    FVectors = (bu32(label=' -- Fur_Vectors')if version>9 else 0)
    FLayers  = (bu32(label=' -- Fur_Layers') if version>9 else 0)
    Materials= (bu32(label=' -- Materials')  if version>7 else 0)
    Nodes    = (bu32(label=' -- Nodes')      if version>7 else 0)
    Objects  = (bu32(label=' -- Objects')    if version>7 else 0)
    Textures = (bu32(label=' -- Textures')   if version>7 else 0)
    Pallets  = (bu32(label=' -- Pallets')    if version>7 else 0)
    Unk3     = (bu32(label=' -- Unknown3')   if version>10 else 0)

    #local header:
    MDL0_Name = Offset_Name(bu32(label=' -- String_Offset'))
    header_len = bu32(      label=' -- Header_Length')
    MDL0_offset = bs32(     label=' -- MDL0_Header_Offset')
    header_unk1 = bu32(     label=' -- Unknown')
    header_unk2 = bu32(     label=' -- Unknown')
    num_verticies = bu32(   label=' -- Vert_Count')
    num_faces = bu32(       label=' -- Face_Count')
    header_unk3 = bu32(     label=' -- Unknown')
    Def_Count = bu32(       label=' -- Def_Count')
    unk_flags = bu32(       label=' -- Unknown_Flags')
    header_unk5 = bu16(     label=' -- Unknown')
    Data_Offset = bu16(     label=' -- Data_Offset')
    boundbox = Bounds(3)
^ very sloppy

the only thing better about the old code is the data sections being vertically smaller.
and that the data is labeled customly for the debug log: (where struct() will label the var names in the log)
Code: Select all
-- importing FitPikachu00.mdl0 --

0x0000000000: read string 'MDL0' of length 4 -- MDL0_Magic
0x0000000004: read 0x0002A160 as 172384 -- Data_Length
0x0000000008: read 0x00000009 as 9 -- Version
0x000000000C: read 0xFFFFFF80 as -128 -- Brres_Header
0x0000000010: read 0x00000518 as 1304 -- Definitions
0x0000000014: read 0x00000560 as 1376 -- Bones
0x0000000018: read 0x00000878 as 2168 -- Vertices
0x000000001C: read 0x000008E0 as 2272 -- Normals
0x0000000020: read 0x00000948 as 2376 -- Colors
0x0000000024: read 0x00000970 as 2416 -- UVs
0x0000000028: read 0x000009D8 as 2520 -- Materials
0x000000002C: read 0x00000A90 as 2704 -- Nodes
0x0000000030: read 0x00000B48 as 2888 -- Objects
0x0000000034: read 0x00000BB0 as 2992 -- Textures
0x0000000038: read 0x00000000 as 0 -- Pallets
0x000000003C: read 0x0002A23C as 172604 -- String_Offset
0x0000000040: jumping to: 0x000002A23B from start of file -- String_Data
0x000002A23B: read 0x0C as 12 -- String_Length
0x000002A23C: read string 'FitPikachu00' of length 12
0x000002A248: jumping to: 0x0000000040 from start of file
0x0000000040: read 0x00000040 as 64 -- Header_Length
0x0000000044: read 0xFFFFFFC0 as -64 -- MDL0_Header_Offset
0x0000000048: read 0x00000000 as 0 -- Unknown
0x000000004C: read 0x00000000 as 0 -- Unknown
0x0000000050: read 0x00001A38 as 6712 -- Vert_Count
0x0000000054: read 0x00001178 as 4472 -- Face_Count
0x0000000058: read 0x00000000 as 0 -- Unknown
0x000000005C: read 0x00000125 as 293 -- Def_Count
0x0000000060: read 0x01010000 as 16842752 -- Unknown_Flags
0x0000000064: read 0x0000 as 0 -- Unknown
0x0000000066: read 0x0040 as 64 -- Data_Offset
 -- Bounds:
0x0000000068: read 0x00000000 as 0.0
0x000000006C: read 0x00000000 as 0.0
0x0000000070: read 0x00000000 as 0.0
0x0000000074: read 0x00000000 as 0.0
0x0000000078: read 0x00000000 as 0.0
0x000000007C: read 0x00000000 as 0.0


so yea, where I need the help on this exactly is getting rid of the order to make it optional instead of required...

here's the code for struct:
Code: Select all
class _STRUCT(object):
    import sys, API as _API
   
    def __init__(_this): pass
    def __call__(parent, size, order, **vars):
        order = order.replace('\n','').replace('\t','').replace(' ','').split(',')

        class _SUB_STRUCT(object):
            _names = vars

            # TODO: get rid of this (perhapse with a hack) (we need a dictionary with the raw order)
            _order = order
            _struct_def = [ vars[s]._struct_def for s in order ] # used for quick comparison in array value-testing

            _addr = None
            _size = size

            def _set_(this, values):
                #print values

                write = bool(len(values))
                skip = this._size
                size = 0

                for vi,name in enumerate(this._order):
                    this.__dict__[name] = this._names[name](values[vi] if write else '')
                    this.__dict__[name].set = this.__dict__[name]._deref_set_
                    if this._size == -1: size += this.__dict__[name]._size
                    else: skip -= this.__dict__[name]._size

                if this._size == -1: this._size=size
                else: parent._API.backend.FILE._current.offset+=skip

            set = None
            def _deref_set_(this, *values):
                '''sets this values pointed to in the file data'''
                this._set_(values)

            def __init__(this, *values):
                #print this._names

                this._addr = parent._API.backend.FILE._current.offset # set the address of this struct
                this._set_(values)
               

        parent.sys.modules['_TypesList'].append(_SUB_STRUCT)
        return _SUB_STRUCT

sys.modules['UGE_Type_Names']['struct'] = 'struct'
global struct; struct = _STRUCT()
sys.modules['UGE_Cons_Names']['structType'] = None
global structType; structType = type

^ if there's something in this code you'd like to know more about, please tell me. :)

sys.modules['UGE_Type_Names'] and sys.modules['UGE_Cons_Names'] is currently only for UMC_SIDE usage...
later on I intend to use these definitions for other things.

I'm sure there's probably a slightly better way than this,
but I'm not a python expert yet and only using what I know to work. :P

EDIT:
it is a long and complex thing, which is why I'm not asking for anything direct, so of course I'm not providing everything. :P
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Tue May 27, 2014 6:03 am

my guess is I did something wrong in explaining my last post...
my second guess is that my system is a bit too complex for anyone here...

------------------------------------------------------------------------------------------------------------------------------

all I'm doing is giving binary files a pointer system,
while also making it extremely easy to work with data between my interface and the file.

because right now, how would normal python programmers import binary data from a file.
as far as I can see, the best method one could recommend would be using the struct module and creating a file object.
where the pointer system would be a complex jump() with offset verification and what not.

with my method, you can simply test your location in the file by getting the pointer to that data.
(instead of adding a number to an offset)

and don't forget, like pointers, you can update the data they link to.

the address for the data is stored with the data and returned when you call ref().
it's actually quite simple, but also extremely useful.

I had to complexify things a bit though to prevent:

data = bu32()
data.set(3000)
^ this will throw an error (NoneType not callable)

however, you can do this:

data = bu32( 0 )
dptr = ref(data)
deref( dptr, bu32 ).set(3000)

which would be the same as in C:

int data = 0
int *dptr = &data
*dptr = 3000

but now imagine your RAM being the binary file you're working on instead.

------------------------------------------------------------------------------------------------------------------------------

all I really need is a good method to get structures working like I've demonstrated below. :)
(or above if you like newest last (me no like to scroll)) :P

does anyone know of any such way I could do this??
(define variables as shown and have them read from the file in given order instead of hashed order)
^ without providing a separate order other than for ignoring certain datas as shown previously.
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby stranac » Tue May 27, 2014 12:25 pm

Tcll wrote:my guess is I did something wrong in explaining my last post...
my second guess is that my system is a bit too complex for anyone here...

My guess (and the reason I don't actually read your posts) is because people think what you're doing is stupid (like the insane code you use), overcomplicated and pointles.

You haven't shown a single actual example you think this is useful for.
Sure, you've shown some code written using your thing, but we have no idea what you're actually doing in that code.
Usually when you post code, you explain it's purpose using terms that mean absolutely nothing in python.

So maybe if you convince us this is useful, getting help would be easier.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1209
Joined: Thu Feb 07, 2013 3:42 pm

Re: UMC memory pointer system revisited

Postby Kebap » Tue May 27, 2014 12:34 pm

Tcll wrote:all I'm doing is giving binary files a pointer system,
while also making it extremely easy to work with data between my interface and the file.

because right now, how would normal python programmers import binary data from a file.


I don't know the answer to that question, as I don't work with binary data. Interesting question though, maybe others can comment more.

I just quoted this part, because I found it easy to understand. Even though I don't understand pointers or C in detail.
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 397
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe

Re: UMC memory pointer system revisited

Postby Tcll » Tue May 27, 2014 3:49 pm

stranac wrote:Sure, you've shown some code written using your thing, but we have no idea what you're actually doing in that code.
Usually when you post code, you explain it's purpose using terms that mean absolutely nothing in python.

So maybe if you convince us this is useful, getting help would be easier.


yep... I figured I'd left out a big load more...
I guess I can't make a post w/o referencing my whole project... grrr

I was trying not to describe because it's not relating to what I need...


what I need is to turn:
Code: Select all
struct( 12, 'a,b,c',
a = bu32,
b = bu32,
c = bu32)

into:
Code: Select all
struct( 12,
a = bu32,
b = bu32,
c = bu32)

^the order man, the order, how do I build something to get rid of the order??
forget about what struct() does, I need to get rid of the order...



if you still want to know...
what UMC does is make things easy to import 3D models (and anything containing or required by them) into it's interface
(from anything (you have to write a script to support the format)) and export them (writing associate export scripts).

at the moment of calling ImportModel() (as described below (or above) this post),
you have already selected a file to import, and a script to run (shown as a filter),
at which this file is loaded into UMC's file-memory system.

the memory for the file data can best be desctibed as array('B', [])
^this is the second fastest, but most stable array I have used for this type of task.

calling one of the functions such as bu32() reads 4 ints from this array and merges them as a bu32 int would be given.
(this goes for big/little signed/unsigned int, any format float, hex, and recently binary data of any supplied byte-size)
^ IEEE754 bf(5) 0x3F80000000 = 1.0
'bu32' is defined as a 'bu_4' object in UMC...
these bu_# objects store a given size of data as an int(), while also storing the long(address) to that data in the file.
(yes, these objects are dynamically created and referenced based on byte-size)

ref() and deref() are the pointer functions.
ref simply returns object._addr for each given object.
deref calls the given object at the given address and updates it's set attribute.

I'm doing all the work of building the data encoders for my scripting system so you don't have to.
all you have to do is read/write the file data.

struct() just allows you to read/write file data in structures

if you'd like to see my full code for the whole thing, I'll have to add you to my box.


if this interface is extremely stupid in how it works (despite it's extreme complexity),
this is what happens when someone has to build a program to help everyone (including noobs),
but yet is given almost no help in learning anything.

yes, I had a long and hard past in building this interface with a high-school algebra1 level education due to my autism.
I couldn't finish their work fast enough for them, so they kept holding me back, so I had to teach myself geometry while I was learning pre-algebra... again...

I'm basically the spit-shine of the earth when it comes to programming... heh
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby Tcll » Tue May 27, 2014 8:47 pm

you know guys... if you guys really thought my code was stupid, you'd tell me of a better way to do my work.
telling me my methods are stupid and not providing any backups to your claims doesn't make you guys look very pro either... :geek:
(if you can't back yourselves up, then don't post until you've studied)

from the way it looks, Blender already does things in the way you guys want me to...
Blender is much too complex to wrap one's head around...
it's just stupid, and not even automated like my system.
while my system IS automated, it's not limited to automation, you can set attributes via Name or ID.

I can't just keep the normal python syntax and not add any functionality, because WITH my current system,
you need about 70% less code (at most) than you would for a Blender script.
User avatar
Tcll
 
Posts: 100
Joined: Wed Jan 01, 2014 6:36 pm

Re: UMC memory pointer system revisited

Postby stranac » Wed May 28, 2014 8:36 am

Tcll wrote:you know guys... if you guys really thought my code was stupid, you'd tell me of a better way to do my work.

We still have NO FUCKING IDEA what you're doing.

"I want to import a binary file" means ABSOLUTELY NOTHING.
All the code you show does some weird stuff with pointers. We have no idea what its real purpose is.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1209
Joined: Thu Feb 07, 2013 3:42 pm

Re: UMC memory pointer system revisited

Postby micseydel » Wed May 28, 2014 9:27 am

+1 to stranac. When I read this, it's like listening to Deepak Chopra talking about quantum physics. Talking about files and RAM in Python the way you seem to be suggests you don't know what you're doing, or at least don't know how to express it.

That said, I suspect that if you took this a bit slower, you'd benefit from our help, and if this is as good as you make it sound, we might benefit too.
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1436
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Next

Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 4 guests