Subject:
Re: [gnupic] Remarks to gpsim of 05/12/2006
From:
Peter ####@####.####
Date:
20 May 2006 20:15:02 +0100
Message-Id: <paqtf.my@zpfd.pg.vpii.pi>
On Sat, 20 May 2006, Scott Dattalo wrote:
>> May I suggest using one of the DBM formats supported natively (no CPAN
>> downloads required) by Perl ? This makes for a *really* easy scripting
>> interface and if possible piping script outputs into the simulator
>> 'live', so one can generate data on-the-fly and must not store it. Perl
>> DBM is one of the fastest ways to store and retrieve large amounts of
>> data with minimum programming effort. The data need not be binary.
>
> Hi Peter,
>
> Can you suggest a particular DBM format? I would want one that is easy for
> users to create data and easy for me to implement. However, I really don't
> want to have a dependency on a third party library (unless it's really
> ubiquitous).
That's the point. The easiest is AnyDBM_File; . Just leave it to Perl to
pick the right one. The input to the process would be from a pipe (there
is no point in duplicating DBM access in gpsim I think). So using the
usual convention, if a stimulus file's first character is a pipe, popen
the file and read its output as a valid stimulus file (using ascii
coding and a really simple syntax (please see below)). The syntax is
negotiable.
As you can see, I advocate a little more than DBM support, but the good
news is it's really simple to support. Also it is not unthinkable to
implement a <anystream>-><stimulus> realtime filter using for example
sox and a perl script in a pipe connected to gpsim using the |program.pl
pipe mechanism I am proposing (in fact it would be tempting to also
implement *output* to a filter, from a pin or other construct - for
obvious reasons - how about testing your dtmf encoder/decoder in real
time, by connecting it to a sound card via pipe output and sox (which
can do lowpass, integration, scaling, yada)). If efficiency matters then
a C program could replace the filter anytime. This is the idea.
Now for some sweat:
So, what I am proposing is:
1. an input for stimulus files of the form '|program' which causes
popen(program, 'r'); and reads stimului from a program, in a simple
ascii format that is to be determined. If there is no pipe then read
from a file, but with the same format ? Reading and writing from plain
files could be mmap()-ed to make this very fast. With the file format I
propose it should run at disk speed anyway. Reading from a program
should be as fast as the program can write out.
2. an output for stimulus/log files of the form '|program' which causes
popen(program, 'w'); and writes output from a selected node, in a simple
ascii format that is to be determined. The read and write formats should
be the same.
It may be that some form of fixed width ascii records will be best to
use, identified/selected by a single header line both for output and for
input. This would not yield particularly small files but they would
compress well so something like zcat could be used easily (both ways).
For example given a sample (hypothetical) file format:
--snip--
# zipzap stimulus header version xyz; [optional comments]
#
# [comments]
#
# columns: ra0 ra1 ra2 rb7
# format: 0.00 C C C
(':'|'+'|'$')TTTTT
1.82 L X H
1.83 L X H
1.84 L X L
# the time has come
(':'|'+'|'$')TTTT1
2.11 H H L
--snap--
where:
- the first line has an id header that identifies the format used,
characters after the ';' are not matched for id.
- optional comments, only at the start of the file
- a columns declarator that assigns each column to a stimulus node, by name
- a format declarator that identifies the format of the data in each
column, the format being fixed by column (think FORTRAN format), using
ascii spaces as separators. This is to make the parser as fast as
possible without using a binary format. Once the format line is read it
is compiled into a list of field readers using either the 'char' method
with offset x into the input line or the 'float' method also with fixed
offset and length. Then any read line is either a comment, a time line
with argument parsed by atoi(), a blank line (first char = whitespace ->
ignore) or a valid data line. Floating point values (unsigned!)
represent volts. Alternately something like AAAA format could be used,
and use ADC counts as input. E.g. AAAAA with 512 as input would mean 2.5
Volts with Vref=5V and 10bit A/D, but 512 would be loaded directly by
gpsim into ADH and ADL.
after this, only data follows, allowing for comments ('#' in col0) and
blank lines (whitespace in col0).
A :TTTT line is a time marker (in ticks), absolute. since reset, a +TTTT
is a time marker relative to the time when the stimulus was started, or
relative to the last seen TTTT line. $TTTT is a time relative to the
last seen data line (think $ as current instruction pointer in .asm).
Examples:
--1--
$TTTT
H L H
--1--
is the same as
--2--
:TTTT
H L H
--2--
and state after reset (before TTTT) is undefined for both.
--3--
# columns: ra0 ra1 ra2
# format: C C C
H H H
:12456
H L H
$8
H H H
$8
H L H
--3--
reset state is H H H, pulses ra1 low for 8 ticks at 12456 absolute, then
high 8 ticks after that (at 12456+8) then low after that (at 12456+8+8),
and remains low. absolute.
--4--
# columns: ra0 ra1 ra2
# format: C C C
H H H
:12456
H L H
+7
H H H
+7
H L H
--4--
as above but different. +7 means 'keep previous state for 7 ticks,
then do what follows'. Basically each stimulus file open should have a
counter that counts relative ticks if not zero, counting down from the
last seen $ or + TTT value, which should be stored (for the next $TTT
to use).
the interpretation of the file is:
inputs are undefined until the first data line is seen. If no time
marker is seen then the data is valid 'immediately', else it is valid
after that time marker is seen. Each data line is data for one tick. If
a time marker is seen as the 'next' line and its time is in the future
(f.ex. a +150 line would put the next data 150 ticks into the future,
after the previous data line was interpreted) then the last data remains
valid until the time is reached. The last data line in the file is valid
'forever'. It should be possible to concatenate data files for input
using a script that does some magic with in-between headers.
There is never a data line with TTTT in it. Comments are allowed (first
char = '#'). There is no such thing as analog output on a PIC (afaik).
When there will be, then there should be output records with analog
values.
For output (future tense) it is necessary for gpsim to 'know' a format.
This could be passed in a config file or even munged into a stimulus
filename using url-like encoding:
(e.g. outstim="|/some/program&format='ra0 ra1 ra2'&columns='C C C'"
or outstim="/some/file&format='ra0 ra1 ra2'&columns='C C C'" )
Note that I just made this up. I hope I did not forget anything
important.
Peter
sorry for the long post, I hope something will come of it.
