gnupic: Thread: Re: [gnupic] COD file format

On 3/2/07, David Barnett ####@####.#### wrote:

> Apparently the COD file format is proprietary ByteCraft information, but gputils
> uses the COD format.  Did we request information from ByteCraft, reverse engineer
> the format, or something else?  I can't see how ByteCraft would release proprietary
> information to us since the source code would be pretty revealing.

I don't know about gputils, but for my software (PICEMU, www.picemulator.com)
I just reverse engineered the format.  It was pretty easy.

Bill

-- 
Psst...  Hey, you... Buddy...  Want a kitten?  straycatblues.petfinder.org

David Barnett wrote:
> Apparently the COD file format is proprietary ByteCraft information, but gputils uses the COD format.  Did we request information from ByteCraft, reverse engineer the format, or something else?  I can't see how ByteCraft would release proprietary information to us since the source code would be pretty revealing.
>
> David
>   
Hi David,

Several years ago when I added COD format support to gpasm, I sought and 
obtained permission from Walter Banks to use ByteCraft's proprietary COD 
format.

Scott

----- Original Message ----- 
From: "Scott Dattalo" ####@####.####
To: ####@####.####
Sent: Friday, March 02, 2007 12:12 PM
Subject: Re: [gnupic] COD file format


> David Barnett wrote:
>> Apparently the COD file format is proprietary ByteCraft information, but 
>> gputils uses the COD format.  Did we request information from ByteCraft, 
>> reverse engineer the format, or something else?  I can't see how 
>> ByteCraft would release proprietary information to us since the source 
>> code would be pretty revealing.
>>
>> David
>>
> Hi David,
>
> Several years ago when I added COD format support to gpasm, I sought and 
> obtained permission from Walter Banks to use ByteCraft's proprietary COD 
> format.
>
> Scott
>
Interesting.  Did they provide any information or did you discover that 
yourself?  Did they restrict how you could use/share that information?  It 
seems like giving permission to use a proprietary format in GPL'd software 
is effectively open-sourcing it.  I don't understand how it can still be 
proprietary.

David 

On Fri, 2007-03-02 at 12:27 -0600, David Barnett wrote:
> ----- Original Message -----
> From: "Scott Dattalo" ####@####.####
>
> Several years ago when I added COD format support to gpasm, I sought and
> > obtained permission from Walter Banks to use ByteCraft's proprietary COD
> > format.
> >
> > Scott
> >
> Interesting.  Did they provide any information or did you discover that
> yourself?  Did they restrict how you could use/share that information?  It
> seems like giving permission to use a proprietary format in GPL'd software
> is effectively open-sourcing it.  I don't understand how it can still be
> proprietary.

Walter provide a document describing the COD file format and supplied a
Pascal program that could parse a .cod file display its contents much like
gpvc does.

I used the Pascal program as a guide to write the C version. I used MPASM
generated outputs to test it. Then I wrote the encoding version. Craig has
since restructured it quite a bit.

I don't know if his allowing us to use the format is 'effectively
open-sourcing' it or not. He only asked that we not distribute his
documentation and source code.

Scott

David Barnett wrote:
> ----- Original Message ----- From: "Scott Dattalo" ####@####.####
> To: ####@####.####
> Sent: Friday, March 02, 2007 12:12 PM
> Subject: Re: [gnupic] COD file format
>
>
>> David Barnett wrote:
>>> Apparently the COD file format is proprietary ByteCraft information, 
>>> but gputils uses the COD format.  Did we request information from 
>>> ByteCraft, reverse engineer the format, or something else?  I can't 
>>> see how ByteCraft would release proprietary information to us since 
>>> the source code would be pretty revealing.
>>>
>>> David
>>>
>> Hi David,
>>
>> Several years ago when I added COD format support to gpasm, I sought 
>> and obtained permission from Walter Banks to use ByteCraft's 
>> proprietary COD format.
>>
>> Scott
>>
> Interesting.  Did they provide any information or did you discover 
> that yourself?  Did they restrict how you could use/share that 
> information?  It seems like giving permission to use a proprietary 
> format in GPL'd software is effectively open-sourcing it.  I don't 
> understand how it can still be proprietary.
You can try at http://www.bytecraft.com/more_info/codfile_request.
I did some time ago, but unfortunately without success :-(  . Probably I 
wasn't persistent enough.
Let me know if you will succeed - I'll try again.

You are mixing the license of the software (gplink, gpsim, ...) and the 
license of the specification (COD file information), which are two 
different things.

Borut

----- Original Message ----- 
From: "Borut Razem" ####@####.####
To: ####@####.####
Sent: Friday, March 02, 2007 3:08 PM
Subject: Re: [gnupic] COD file format


> You can try at http://www.bytecraft.com/more_info/codfile_request.
> I did some time ago, but unfortunately without success :-(  . Probably I 
> wasn't persistent enough.
> Let me know if you will succeed - I'll try again.
Yes, I've seen that form before.  For now I'm just curious...I don't have 
any real need to see it, so there's nothing for me to put on the request 
form.  I'll let you know if I do try and succeed, but that probably wouldn't 
be any time soon.

> You are mixing the license of the software (gplink, gpsim, ...) and the 
> license of the specification (COD file information), which are two 
> different things.
I wasn't aware there could be a separate license for the specification.  I 
figured that describing the file format to someone else would violate the 
agreement, and I thought that the source code itself is a pretty effective 
way to describe the file format.  I didn't see the point of telling everyone 
the format is proprietary if the next best thing to a format specification 
was publicly available.  I guess there's a gray area in how 'derived' the 
information is.

Thanks for the info.

David 

On Fri, 02 Mar 2007 10:12:24 -0800
Scott Dattalo ####@####.#### wrote:

> Hi David,
> 
> Several years ago when I added COD format support to gpasm, I sought
> and obtained permission from Walter Banks to use ByteCraft's
> proprietary COD format.

Since you know something about the COD format, did you see the
discussion between Robert and me under "Object File Bit
Relocation" (1/26/07)?  In a nutshell, I was wondering whether there
would be any technical complications to relocating bit indexes (eg.
"bsf my_reg, relocatable_index".  Do you have any idea on that?

David

David wrote:
> On Fri, 02 Mar 2007 10:12:24 -0800
> Scott Dattalo ####@####.#### wrote:
>
>   
>> Hi David,
>>
>> Several years ago when I added COD format support to gpasm, I sought
>> and obtained permission from Walter Banks to use ByteCraft's
>> proprietary COD format.
>>     
>
> Since you know something about the COD format, did you see the
> discussion between Robert and me under "Object File Bit
> Relocation" (1/26/07)?  In a nutshell, I was wondering whether there
> would be any technical complications to relocating bit indexes (eg.
> "bsf my_reg, relocatable_index".  Do you have any idea on that?
>   

I saw the discussion on the relocatable bit types, but didn't take time 
to respond.

Probably the best way to support relocatable bits is by introducing a 
native bit type to gpasm/MPASM. The linker can pack the bit together in 
bytes. Having a native bit type could also make the standard header bit 
definition usage less error prone. For example, suppose in gpasm we wrote:

bLATA0  bit LATA, 0

instead of

RA0 equ 0

and then in the actual code we could write:

    BSF  bLATA0
instead of
    BSF LATA, RA0

(Maybe using a peripheral bit as an example would have been better).

For dynamically allocated bits we could write:

bFlag0  dbit 1
bFlag1  dbit 1
etc.

Then the linker can pack all of the bit definitions into bytes.

I could envision other extensions as well. For example, you could define 
bit fields that consist of 2 or more bits. You might have to introduce a 
concept of linker defined bit masks for accessing the bit fields. For 
example:

myBitField dbit 4    ; define a 4-bit variable

    MOVLW  ~MASK(myBitField)
    ANDWF  myBitField, W

The only thing I don't like about this approach is that it adds a 
non-MPASM supported extension. Perhaps we could introduce the extension 
in the form of macros that gpasm knows how to optimize? Then an 
assembler flag could be used to turn off any kind of gpasm extensions.

BTW, since SDCC supports a bit type, I had added support for bit fields 
in the pCode. At one point (and maybe it's still the case), you could 
allocate a bitfield and SDCC would pack bits into it. There wasn't much 
in the way of trying to optimize this allocation though.

Scott

----- Original Message ----- 
From: "Scott Dattalo" ####@####.####
To: ####@####.####
Sent: Wednesday, March 07, 2007 11:14 AM
Subject: Re: [gnupic] COD file format


> David wrote:
>>
>> Since you know something about the COD format, did you see the
>> discussion between Robert and me under "Object File Bit
>> Relocation" (1/26/07)?  In a nutshell, I was wondering whether there
>> would be any technical complications to relocating bit indexes (eg.
>> "bsf my_reg, relocatable_index".  Do you have any idea on that?
>>
>
> I saw the discussion on the relocatable bit types, but didn't take time to 
> respond.
>
> Probably the best way to support relocatable bits is by introducing a 
> native bit type to gpasm/MPASM. The linker can pack the bit together in 
> bytes. Having a native bit type could also make the standard header bit 
> definition usage less error prone. For example, suppose in gpasm we wrote:
>
> bLATA0  bit LATA, 0
>
> instead of
>
> RA0 equ 0
>
> and then in the actual code we could write:
>
>    BSF  bLATA0
> instead of
>    BSF LATA, RA0
>
> (Maybe using a peripheral bit as an example would have been better).
I think I see what you're saying.  The difference between those would be the 
same as between these:
   #define bLATA0 LATA, 0
   BSF bLATA0
and
   RA0 equ 0
   BSF LATA, RA0

There are times when you need each.  The advantage of the first is that you 
can't accidentally get the right bit from the wrong register like "BSF LATB, 
RA0".  But notice the first form can be easily derived from the second and 
not vice versa.  I usually define both forms for each bit when I'm working 
in assembly.

<snip>
> I could envision other extensions as well. For example, you could define 
> bit fields that consist of 2 or more bits. You might have to introduce a 
> concept of linker defined bit masks for accessing the bit fields. For 
> example:
>
> myBitField dbit 4    ; define a 4-bit variable
>
>    MOVLW  ~MASK(myBitField)
>    ANDWF  myBitField, W
If you had the indices, you could easily create the mask:
   MOVLW (1 << (hi_index(myBitField) + 1)) - (1 << low_index(myBitField))
Maybe "easily" is an exaggeration =).

> The only thing I don't like about this approach is that it adds a 
> non-MPASM supported extension.
I definitely agree there.  Back to the different representations of a bit's 
address, we would need the register and bit offset for each bit, and it 
would be convenient to access them as one address at times (like bLATA0). 
Also, since we will have to introduce at least some variant of dbit to 
gpasm, the best we can do without cooperation from Microchip is to localize 
the differences so that porting between gpasm and MPASM requires few 
modifications.  One way to do that would be to mangle the bit names to 
something like this:
    myBit dbit 1
    myBitField dbit 4
    ...
    bsf my_Bit
    movlw (1 << i_myBit)    ; index (offset) of myBit
    andwf r_myBit, w    ; register address of myBit
    movlw (1 << (ih_myBitField + 1)) - (1 << il_myBitField)    ; high and 
low indices of myBitField
That way to port the code to MPASM you could replace the definitions:
    myBit dbit 1
    myBitField dbit 4
with something like this:
    res r_myBit 0
    res r_myBitField 0
    res _field1 1
    i_myBit EQU 0
    il_myBitField EQU 1
    ih_myBitField EQU 4
    #define myBit r_myBit, i_myBit
    #define myBitField ????
We'd also have to document the reserved name formats for variables so that 
portable code could avoid using them for other purposes.  The bit field 
would take special consideration, but since (a) individual bits tend to 
outnumber bit fields and (b) 1-register-per-bit wastes more space than 
non-packed bit fields, I think individual bits are a much higher priority.

Still, complete compatibility would be really nice, so maybe we could agree 
with Microchip on a format they could support in the future.

Anyway, we can discuss how to implement it later...I was really just 
wondering whether it was feasible, and it sounds like you believe it is. 
Thanks for all the info!

David 

All,

In response to David Barnett's question this should clear up
whatever misconceptions there may be over the use of
Byte Craft's  .COD file format.


Scott Dattalo contacted me at some point and asked me
what the status of the .COD file format was?
He pointing out the sources for the tools he was
writing would be openly available. My response
was the .COD file information could be used
(no problem at all) the information was copyright
Byte Craft and documentation should refer to
Byte Craft for details rather than reproduce the
.COD file document.

The reason is, we did not want to see the standards
for this format "evolve" the way many other debugging
information standards have with many "standard
variations".

Byte Craft's policy related to copies of the .COD
file format is

  The .COD file and support tools (including sources for a
  .COD file viewer) are emailed to anyone who asks without
  charge. We maintain the standard and from time to time
  make additions to it in co-operation to those who are
  using the .COD file.

  Byte Craft provides support for the .COD file information
  to developers and interested parties. We are interested in
  knowing who is using the .COD file to encourage inter operability
  between development tools for embedded systems.

> You can try at http://www.bytecraft.com/more_info/codfile_request.
> I did some time ago, but unfortunately without success :-(  . Probably I
> wasn't persistent enough.)

I just checked Borut you contacted us on 3 Feb 2006  and I
couldn't find any  trace of us sending a copy to you. I just did.

The form on Byte Crafts web site asks basic contact information
only. COD information is usually sent out within a day or two.

Bill, I could have saved you some time :)

The .COD file is being extended to support multiple
processor execution environments and many new data types.

The .COD file standard is version upward compatible a
critical feature that has made mismatched tools still work
well together.

Contact me directly if you have any specific questions

Walter Banks
Byte Craft Limited

========================================
Borut Razem wrote:

> David Barnett wrote:
> > ----- Original Message ----- From: "Scott Dattalo" ####@####.####
> > To: ####@####.####
> > Sent: Friday, March 02, 2007 12:12 PM
> > Subject: Re: [gnupic] COD file format
> >
> >
> >> David Barnett wrote:
> >>> Apparently the COD file format is proprietary ByteCraft information,
> >>> but gputils uses the COD format.  Did we request information from
> >>> ByteCraft, reverse engineer the format, or something else?  I can't
> >>> see how ByteCraft would release proprietary information to us since
> >>> the source code would be pretty revealing.
> >>>
> >>> David
> >>>
> >> Hi David,
> >>
> >> Several years ago when I added COD format support to gpasm, I sought
> >> and obtained permission from Walter Banks to use ByteCraft's
> >> proprietary COD format.
> >>
> >> Scott
> >>
> > Interesting.  Did they provide any information or did you discover
> > that yourself?  Did they restrict how you could use/share that
> > information?  It seems like giving permission to use a proprietary
> > format in GPL'd software is effectively open-sourcing it.  I don't
> > understand how it can still be proprietary.

>
> You can try at http://www.bytecraft.com/more_info/codfile_request.
> I did some time ago, but unfortunately without success :-(  . Probably I
> wasn't persistent enough.
> Let me know if you will succeed - I'll try again.
>
> You are mixing the license of the software (gplink, gpsim, ...) and the
> license of the specification (COD file information), which are two
> different things.
>
> Borut
> Walter provide a document describing the COD file format and supplied a
> Pascal program that could parse a .cod file display its contents much like
> gpvc does.
>
> I used the Pascal program as a guide to write the C version. I used MPASM
> generated outputs to test it. Then I wrote the encoding version. Craig has
> since restructured it quite a bit.
>
> I don't know if his allowing us to use the format is 'effectively
> open-sourcing' it or not. He only asked that we not distribute his
> documentation and source code.
>
> Scott