gnupic: Thread: Re: [gnupic] Remarks to gpsim of 05/12/2006

On Fri, 2006-05-19 at 14:01 +0200, Tobias Schlottke wrote:

> I downloaded the top svn version at 05/12/2006 and
> tested with the following small program:

<snip>


> It sets bit 0 of PortA as input,
> sets bit 0 of PortA to 1 and
> checks the state of bit 0.
>
> Right now its evaluated to 1, so movlw 53 is executed.
> That is not what I've expected. By default all inputs
> are read as 0, so even if I try to overwrite the value
> it should not change.
>
> But:
> ====
> If I connect a stimulus to the port gpsim behaves as
> expected, the bit is sampled as 0. Unfortunally the
> change of the state of the port is not visible in the
> breadboard window. So gpsim behaves like the real
> world: undefined pins are evil....

Hi Toby,

gpsim makes an attempt to accurately model I/O pin states: even the
unknown ones. The simulation engine (which is separate from the gui)
contains a "bit state" for I/O pins. You can examine the state with the
stimulus command at the command line. Currently these are the bit states
supported:

  H - High
  L - Low
  W - Weak pullup
  w - Weak pulldown
  Z - High impedance
  X - unknown

Part of the confusion I think is that the gui really only conveys the 'H'
and 'L' states and not the others. In your particular case, the I/O pin
for porta 0 is in the Z state. The gui (apparently) interprets this as a
low. When your program reads the I/O port it interprets this as a high.
Which is right? I don't know, but I do know that the lack of consistency
between the two cannot be correct!

Of course, when you connect a stimulus to the I/O pin the ambiguity is
resolved.

BTW, gpsim can even model an unknown bit state. However this is not fully
supported for the PIC processors.

Scott

Hi Scott,

thank you for your response.

On Fri, 19 May 2006, Scott Dattalo wrote:

.....

> BTW, gpsim can even model an unknown bit state. However this is not fully
> supported for the PIC processors.

?? I dont understand this. If a bit state is unknown 
and the program tests the bit, what should be the 
result? Ok, it could break with a warning or with an 
error.

I have two further questions:
It still regards to gpsim of 05/12/2006 so I dont open 
a new thread.

First:
In file src/stimuli.cc method
  ValueStimulus::start() (around line 1650):

     current       = initial.v;
     next_sample   = *sample_iterator;
     future_cycle  = next_sample.time + start_cycle;

     get_cycles().set_break(future_cycle, this);

   }

I appended (uncommented) the "+ start_cycle" for the 
future_cycle computation. IMHO all times are relative 
to the start_cycle.

Do you agree?

Second:
In the same file, same method at the beginning of the 
method you add an initial data set to the array if the 
period != 0. With this special handling I am not able 
to create an exact stimulus with an exact repetition. I 
get glitches if the period is reached and the stimulus 
rolls over. May be because the pointer into the data 
set is always moved forward one step. If we have 
multiple values (even equal ones) for the same time, 
the mechanism gets confused. I dont understand 
this extra value. If the last point of the array is 
different from the initial state who cares?

Ah!

After thinking again I found the following:
If you specify the stimulus as:

{
10, 0,
10, 1,
10, 0,
10, 1,
10, 0,
10, 1,
10, 0,
20, 1
}

you'll get a 3 time toggling bit. The rollover 
mechanism inserts for same time two data points. These 
are interpreted sequentially in two different cycles. 
So the problem is not the insertion of an additional 
data point, instead we should skip all points until we 
get a time stamp in the future.
But right now I'm not able to do this.


Best regards
Toby

Tobias Schlottke wrote:

>> BTW, gpsim can even model an unknown bit state. However this is not fully
>> supported for the PIC processors.
> 
> 
> ?? I dont understand this. If a bit state is unknown and the program 
> tests the bit, what should be the result? Ok, it could break with a 
> warning or with an error.

That's one behavior (and is what I do). However, this behavior is 
extended beyond what I think you may be expecting.

For example, consider something like this:

   ; interrupt routine

       MOVF  PORTA,W
       MOVWF SampledPORTA

   ;------------


   ; non-interrupt code:

   ; Test the I/O state capture in the interrupt routine:

      BTFSS  SampledPORTA,0
       CALL  RA0_went_low


This code copies the state of the I/O port to a register and tests it at 
a later time. With the unknown/3-state propagation, the unknown state is 
stored in the variable SampledPORTA. If you attempt to execute condition 
code based on the state of an unknown bit then the simulation halts. You 
can examine the trace log to determine when the register obtained it's 
unknown state.

But, as I said, the PIC port of gpsim does not fully support this feature.


> 
> I have two further questions:
> It still regards to gpsim of 05/12/2006 so I dont open a new thread.
> 
> First:
> In file src/stimuli.cc method
>  ValueStimulus::start() (around line 1650):
> 
>     current       = initial.v;
>     next_sample   = *sample_iterator;
>     future_cycle  = next_sample.time + start_cycle;
> 
>     get_cycles().set_break(future_cycle, this);
> 
>   }
> 
> I appended (uncommented) the "+ start_cycle" for the future_cycle 
> computation. IMHO all times are relative to the start_cycle.
> 
> Do you agree?

I believe so. I made some changes here recently and may've broken this.

> Second:
> In the same file, same method at the beginning of the method you add an 
> initial data set to the array if the period != 0. With this special 
> handling I am not able to create an exact stimulus with an exact 
> repetition. I get glitches if the period is reached and the stimulus 
> rolls over. May be because the pointer into the data set is always moved 
> forward one step. If we have multiple values (even equal ones) for the 
> same time, the mechanism gets confused. I dont understand this extra 
> value. If the last point of the array is different from the initial 
> state who cares?

I think the asynchronous stimuli are too confusing. This past week I've 
begun working on a new stimulus mechanism. This new stimulus will 
actually be a collection of dynamically loadable modules. Think of them 
as function generators!

I'm working on one called 'pulsegen' that will be used for generating 
pulses. It has the same functionality of the asynchronous stimulus, 
however it's behavior is controlled via editable attributes. The 
asynchronous stimulus requires all of it's data in one giant gpsim 
command and is uneditable.

There are attributes to control it's electrical behavior, e.g. output 
resistance, output capacitance, drive voltage, etc. And there attributes 
to control its personality. For example there are .set, .clear, and 
.delete that have the units of simulation cycles. Here's how you use them:


  module load pulsegen V1
  V1.set = 100   # drive high at cycle 100
  V1.clear = 200 # drive low at cycle 100

If you need to insert a pulse, then you can do this:

  V1.set =50
  V1.clear = 52

If you decide that you need to make a pulse wider:

  V1.delete = 52
  V1.clear = 55

The set and clear attributes remember the last time written to them. So 
you can do this:

  symbol PulseStart=100   # create a symbol with a value
  symbol PulseWidth=50

  V1.set = PulseStart
  V1.clear = V1.set + PulseWidth

The point I'm trying to make is that attributes give you quite bit of 
flexibility.

In addition to the pulse generator, I'm going to create something like 
SPICE's PWL (piece-wise linear) voltage source. Essentially, this will 
ramp between the data points instead of stepping. This will allow 
triangle waves and saw tooth waves.


> 
> Ah!
> 
> After thinking again I found the following:
> If you specify the stimulus as:
> 
> {
> 10, 0,
> 10, 1,
> 10, 0,
> 10, 1,
> 10, 0,
> 10, 1,
> 10, 0,
> 20, 1
> }

That won't work because you're repeatedly specifying cycle 10.

Scott

On Fri, 19 May 2006, Scott Dattalo wrote:

> I think the asynchronous stimuli are too confusing. This past week I've begun 
> working on a new stimulus mechanism. This new stimulus will actually be a 
> collection of dynamically loadable modules. Think of them as function 
> generators!

Mmmh, may be confusing but they work. I'm now observing 
a new problem. I have big stimulus files with kind of 
serial bit patterns. One file has 100,000 entries of 
transistions. Size about 3 MB. This file is of course 
generated by a script. In version 0.21.2 I had no 
problems with this file. Load time was ok and the 
simulation was fine.

With the actual top version it seems to me that there 
is a kind of quadratic cost in reading this file. If I 
use more than 25,000 cpu load goes to 90% and the 
reading speed gets slower and slower. I checked this 
with strace/truss. Ok, I split the data into 4 stimuli 
and attached all of them to one node. I thought that a 
stimulus goes into high impedance mode if no values are 
left. Again I was trapped by the breadboard. The color 
of the pin changed exactly at the expected timestamps 
but then I double checked with the output of "stimulus" 
(Thanks again for the hint!). It seems that all the 4 
generators a coupled with a resistor network and the 
the result is an analog voltage which can not be 
sampled properly. I specified "digital" mode.

Do you have a hint for me how to
improve the reading of big stimulus files?
or how to couple stimuli with kind of a multiplexer?


> I'm working on one called 'pulsegen' that will be used for generating pulses. 
> It has the same functionality of the asynchronous stimulus, however it's 
> behavior is controlled via editable attributes. The asynchronous stimulus 
> requires all of it's data in one giant gpsim command and is uneditable.
>
> There are attributes to control it's electrical behavior, e.g. output 
> resistance, output capacitance, drive voltage, etc. And there attributes to 
> control its personality. For example there are .set, .clear, and .delete that 
> have the units of simulation cycles. Here's how you use them:
>
>
....
>
> In addition to the pulse generator, I'm going to create something like 
> SPICE's PWL (piece-wise linear) voltage source. Essentially, this will ramp 
> between the data points instead of stepping. This will allow triangle waves 
> and saw tooth waves.

On one hand side this is nice. On the other hand I 
prefer a stupid ascii format what I can use for 
machine generated files. And there is always a case 
that's too complex for the internal generators and then 
we move again to precomputed or measured data. I.E. 
you want to simulate a simple speech recognition 
system. You have recorded voice samples and want to 
simulate the behaviour of your program with these 
samples. There is no other way than just reading the 
files.

Just my to cents.

Best regards
Toby

Tobias Schlottke wrote:
> On Fri, 19 May 2006, Scott Dattalo wrote:
> 
>> I think the asynchronous stimuli are too confusing. This past week 
>> I've begun working on a new stimulus mechanism. This new stimulus will 
>> actually be a collection of dynamically loadable modules. Think of 
>> them as function generators!
> 
> 
> Mmmh, may be confusing but they work. I'm now observing a new problem. I 
> have big stimulus files with kind of serial bit patterns. One file has 
> 100,000 entries of transistions. Size about 3 MB. This file is of course 
> generated by a script. In version 0.21.2 I had no problems with this 
> file. Load time was ok and the simulation was fine.

100,000! That's a lot. When I'm confronted with a situation such as this 
I'll write a special module to cope with it. However, I think that's too 
complicated for gpsim users. It sounds like what may be more appropriate 
is to have something like a "file stimulus". The stimulus takes a file 
name as it's input and will fetch data only when it's needed.

The reason that the existing stimuli take longer now than they use to is 
that they're more generic. In the past, you could only have 0's and 1's 
for data stream. Now you can have any kind of type. This requires more 
memory storage.


> Do you have a hint for me how to
> improve the reading of big stimulus files?
> or how to couple stimuli with kind of a multiplexer?

Let me design a file stimulus. I have another application I can use this 
for too.


>> In addition to the pulse generator, I'm going to create something like 
>> SPICE's PWL (piece-wise linear) voltage source. Essentially, this will 
>> ramp between the data points instead of stepping. This will allow 
>> triangle waves and saw tooth waves.
> 
> 
> On one hand side this is nice. On the other hand I prefer a stupid ascii 
> format what I can use for machine generated files. And there is always a 
> case that's too complex for the internal generators and then we move 
> again to precomputed or measured data. I.E. you want to simulate a 
> simple speech recognition system. You have recorded voice samples and 
> want to simulate the behaviour of your program with these samples. There 
> is no other way than just reading the files.

There's nothing preventing you from putting the pulsegen configuration 
into a configuration script!

Scott

On Fri, 19 May 2006, Scott Dattalo ####@####.#### wrote :
>
>Let me design a file stimulus. I have another application I can use 
>this for too.

One thought - make sure you design it such that it will be happy to be 
fed a named pipe (I wouldn't expect that to be too hard) because that 
gives the user extra flexibility to generate the stimulus "on the fly" 
by a simple external program.
-- 
Rob Pearce                       http://www.bdt-home.demon.co.uk

The contents of this | Sometimes a man who deserves to be looked down upon
message are purely   | because he is a fool is despised only because he is a
my opinion. Don't    | lawyer.
believe a word.      |  -- Montesquieu

On Fri, 19 May 2006, Scott Dattalo wrote:

...

> complicated for gpsim users. It sounds like what may be more appropriate is 
> to have something like a "file stimulus". The stimulus takes a file name as 
> it's input and will fetch data only when it's needed.

Yes, that sounds cool. Thanks in advance!

Best regards
Toby

> On Fri, 19 May 2006, Scott Dattalo wrote:
>> complicated for gpsim users. It sounds like what may be more appropriate is 
>> to have something like a "file stimulus". The stimulus takes a file name as 
>> it's input and will fetch data only when it's needed.

May I suggest using one of the DBM formats supported natively (no CPAN 
downloads required) by Perl ? This makes for a *really* easy scripting 
interface and if possible piping script outputs into the simulator 
'live', so one can generate data on-the-fly and must not store it. Perl 
DBM is one of the fastest ways to store and retrieve large amounts of 
data with minimum programming effort. The data need not be binary.

Peter

> May I suggest using one of the DBM formats supported natively (no CPAN
> downloads required) by Perl ? This makes for a *really* easy scripting
> interface and if possible piping script outputs into the simulator
> 'live', so one can generate data on-the-fly and must not store it. Perl
> DBM is one of the fastest ways to store and retrieve large amounts of
> data with minimum programming effort. The data need not be binary.

Hi Peter,

Can you suggest a particular DBM format? I would want one that is easy for
users to create data and easy for me to implement. However, I really don't
want to have a dependency on a third party library (unless it's really
ubiquitous).

Scott

On Sat, 20 May 2006, Scott Dattalo wrote:

>> May I suggest using one of the DBM formats supported natively (no CPAN
>> downloads required) by Perl ? This makes for a *really* easy scripting
>> interface and if possible piping script outputs into the simulator
>> 'live', so one can generate data on-the-fly and must not store it. Perl
>> DBM is one of the fastest ways to store and retrieve large amounts of
>> data with minimum programming effort. The data need not be binary.
>
> Hi Peter,
>
> Can you suggest a particular DBM format? I would want one that is easy for
> users to create data and easy for me to implement. However, I really don't
> want to have a dependency on a third party library (unless it's really
> ubiquitous).

That's the point. The easiest is AnyDBM_File; . Just leave it to Perl to 
pick the right one. The input to the process would be from a pipe (there 
is no point in duplicating DBM access in gpsim I think). So using the 
usual convention, if a stimulus file's first character is a pipe, popen 
the file and read its output as a valid stimulus file (using ascii 
coding and a really simple syntax (please see below)). The syntax is 
negotiable.

As you can see, I advocate a little more than DBM support, but the good 
news is it's really simple to support. Also it is not unthinkable to 
implement a <anystream>-><stimulus> realtime filter using for example 
sox and a perl script in a pipe connected to gpsim using the |program.pl 
pipe mechanism I am proposing (in fact it would be tempting to also 
implement *output* to a filter, from a pin or other construct - for 
obvious reasons - how about testing your dtmf encoder/decoder in real 
time, by connecting it to a sound card via pipe output and sox (which 
can do lowpass, integration, scaling, yada)). If efficiency matters then 
a C program could replace the filter anytime. This is the idea.

Now for some sweat:

So, what I am proposing is:

1. an input for stimulus files of the form '|program' which causes 
popen(program, 'r'); and reads stimului from a program, in a simple 
ascii format that is to be determined. If there is no pipe then read 
from a file, but with the same format ? Reading and writing from plain 
files could be mmap()-ed to make this very fast. With the file format I 
propose it should run at disk speed anyway. Reading from a program 
should be as fast as the program can write out.

2. an output for stimulus/log files of the form '|program' which causes 
popen(program, 'w'); and writes output from a selected node, in a simple 
ascii format that is to be determined. The read and write formats should 
be the same.

It may be that some form of fixed width ascii records will be best to 
use, identified/selected by a single header line both for output and for 
input. This would not yield particularly small files but they would 
compress well so something like zcat could be used easily (both ways). 
For example given a sample (hypothetical) file format:

--snip--
# zipzap stimulus header version xyz; [optional comments]
# 
# [comments]
#
# columns: ra0 ra1 ra2 rb7
# format: 0.00  C  C  C
(':'|'+'|'$')TTTTT
1.82  L  X  H
1.83  L  X  H
1.84  L  X  L

# the time has come
(':'|'+'|'$')TTTT1
2.11  H  H  L
--snap--

where:

- the first line has an id header that identifies the format used, 
characters after the ';' are not matched for id.

- optional comments, only at the start of the file

- a columns declarator that assigns each column to a stimulus node, by name

- a format declarator that identifies the format of the data in each 
column, the format being fixed by column (think FORTRAN format), using 
ascii spaces as separators. This is to make the parser as fast as 
possible without using a binary format. Once the format line is read it 
is compiled into a list of field readers using either the 'char' method 
with offset x into the input line or the 'float' method also with fixed 
offset and length. Then any read line is either a comment, a time line 
with argument parsed by atoi(), a blank line (first char = whitespace -> 
ignore) or a valid data line. Floating point values (unsigned!) 
represent volts. Alternately something like AAAA format could be used, 
and use ADC counts as input. E.g. AAAAA with 512 as input would mean 2.5 
Volts with Vref=5V and 10bit A/D, but 512 would be loaded directly by 
gpsim into ADH and ADL.

after this, only data follows, allowing for comments ('#' in col0) and 
blank lines (whitespace in col0).

A :TTTT line is a time marker (in ticks), absolute. since reset, a +TTTT 
is a time marker relative to the time when the stimulus was started, or 
relative to the last seen TTTT line. $TTTT is a time relative to the 
last seen data line (think $ as current instruction pointer in .asm).

Examples:

--1--
$TTTT
H L H
--1--

is the same as

--2--
:TTTT
H L H
--2--

and state after reset (before TTTT) is undefined for both.

--3--
# columns: ra0 ra1 ra2
# format: C C C
H H H
:12456
H L H
$8
H H H
$8
H L H
--3--

reset state is H H H, pulses ra1 low for 8 ticks at 12456 absolute, then 
high 8 ticks after that (at 12456+8) then low after that (at 12456+8+8), 
and remains low. absolute.

--4--
# columns: ra0 ra1 ra2
# format: C C C
H H H
:12456
H L H
+7
H H H
+7
H L H
--4--

as above but different. +7 means 'keep previous state for 7 ticks, 
then do what follows'. Basically each stimulus file open should have a 
counter that counts relative ticks if not zero, counting down from the 
last seen $ or + TTT value, which should be stored (for the next $TTT 
to use).

the interpretation of the file is:

inputs are undefined until the first data line is seen. If no time 
marker is seen then the data is valid 'immediately', else it is valid 
after that time marker is seen. Each data line is data for one tick. If 
a time marker is seen as the 'next' line and its time is in the future 
(f.ex. a +150 line would put the next data 150 ticks into the future, 
after the previous data line was interpreted) then the last data remains 
valid until the time is reached. The last data line in the file is valid 
'forever'. It should be possible to concatenate data files for input 
using a script that does some magic with in-between headers.

There is never a data line with TTTT in it. Comments are allowed (first 
char = '#'). There is no such thing as analog output on a PIC (afaik). 
When there will be, then there should be output records with analog 
values.

For output (future tense) it is necessary for gpsim to 'know' a format. 
This could be passed in a config file or even munged into a stimulus 
filename using url-like encoding:

(e.g. outstim="|/some/program&format='ra0 ra1 ra2'&columns='C C C'"
or    outstim="/some/file&format='ra0 ra1 ra2'&columns='C C C'" )

Note that I just made this up. I hope I did not forget anything 
important.

Peter

sorry for the long post, I hope something will come of it.