From bil at ccrma.Stanford.EDU  Mon Aug  8 02:40:26 2022
From: bil at ccrma.Stanford.EDU (bil at ccrma.Stanford.EDU)
Date: Mon, 08 Aug 2022 02:40:26 -0700
Subject: [CM] Snd 22.6
Message-ID: <e32f4070499bc82337aeb745f031807e@ccrma.stanford.edu>

Snd 22.6

Mostly fixing bugs and making minor optimizations.

s7: (random-state) returns the current random-state object.
     copy can handle random-state objects (but in s7/gmp it's a no-op)

checked: sbcl 2.2.7

Thanks!:  Anders Vinjar


From zmoelnig at iem.at  Tue Aug 16 08:49:17 2022
From: zmoelnig at iem.at (IOhannes m zmoelnig)
Date: Tue, 16 Aug 2022 17:49:17 +0200
Subject: [CM] snd on s390x (probably a BigEndian issue)
Message-ID: <9f93da1c-5bdd-e2d9-af54-359bce8dabac@iem.at>

hi,

I'm packaging 'snd' for Debian (and derivatives, like Ubuntu).

as of snd-22.6, i've added some basic tests that are run on your 
(Debian's) CI-infrastructure.

this has promptly triggered a problem on one of your build 
architectures, namely s390x (aka "IBM System/390" a big-endian, 
mainframe-class architecture).

the test is basically running this minimal s7 script:
```s7
(new-sound)
(delete-samples 0 10000)
(pad-channel 0 44100)
(set! (sample 10) 0.5)
(save-sound-as "test.wav" 0 :header-type mus-riff :sample-type mus-lshort)
(exit 0)
```

unfortunately, `snd` segfaults :-(

it also segfaults, when i just run "echo | snd", so i gather, the 
problem is not with the script.

the "snd" variant used for these tests are compiled without any GUI (not 
even notcurses), but with JACK and ALSA support.

both tests also *succeed* on amd64 (aka "x86_64"), i386 (aka "x86"), 
armel (aka "armv4" or similar), armhf (aka "armv6"; or your beloved 
rasbperry), "arm64" (64bit arm, e.g. RPi4) and "ppcel" (whatever).

none of the test machine has a soundcard installed (or jackd running).

afaict, all succeeding architectures are Little Endian (though this 
might just be a red herring).

the full logs of the failing tests can be seen on [1], but the 
re-curring error-message is:
 > s7_make_circular_signature got too few entries: <unknown object! 
type: messed up object? (64), opt_op: 256 safe_closure_a_o, flags: 
#x4000000001000001 immutable unheap>


any ideas?

gfmasdr
IOhannes



[1] 
<https://ci.debian.net/data/autopkgtest/testing/s390x/s/snd/24661415/log.gz>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://cm-mail.stanford.edu/pipermail/cmdist/attachments/20220816/7f4220d6/attachment.sig>

From bil at ccrma.Stanford.EDU  Tue Aug 16 09:57:42 2022
From: bil at ccrma.Stanford.EDU (bil at ccrma.Stanford.EDU)
Date: Tue, 16 Aug 2022 09:57:42 -0700
Subject: [CM] snd on s390x (probably a BigEndian issue)
In-Reply-To: <9f93da1c-5bdd-e2d9-af54-359bce8dabac@iem.at>
References: <9f93da1c-5bdd-e2d9-af54-359bce8dabac@iem.at>
Message-ID: <cf3bb5d4995d759b4abe4fdb9b79925e@ccrma.stanford.edu>

Thanks for the bug report.  Is this segfault new in version 22.6?  
Wikipedia
says the system/390 was a 32-bit machine, and it's possible s7 doesn't 
notice --
I depend on compiler macros like SIZEOF_VOID_P.  It might help to see
the configure output or mus-config.h or a stacktrace at the error.


From bil at ccrma.Stanford.EDU  Tue Aug 16 13:25:13 2022
From: bil at ccrma.Stanford.EDU (bil at ccrma.Stanford.EDU)
Date: Tue, 16 Aug 2022 13:25:13 -0700
Subject: [CM] snd on s390x (probably a BigEndian issue)
Message-ID: <fbace2ad46c09c11fdc887718fadb1ba@ccrma.stanford.edu>

The error is hit in init_rootlet before the error_hook is set up,
so it segfaults when it tries to call the error_hook.  The line
number given in init_roolet is bogus, so I can't tell where it is.
Actually a lot of stuff is messed up.  I guess we'll just have
to disappoint the IBM mainframe guys.  Back in 1969 (was
it really more than 50 years ago?) I wrote some code (I
think it was Basic, maybe Fortran) on an IBM mainframe
(probably a 360?) -- I vaguely remember the typewriter
(was it a typewriter?) had round keys??

From zmoelnig at iem.at  Wed Aug 17 01:01:28 2022
From: zmoelnig at iem.at (IOhannes m zmoelnig)
Date: Wed, 17 Aug 2022 10:01:28 +0200
Subject: [CM] snd on s390x (probably a BigEndian issue)
In-Reply-To: <fbace2ad46c09c11fdc887718fadb1ba@ccrma.stanford.edu>
References: <fbace2ad46c09c11fdc887718fadb1ba@ccrma.stanford.edu>
Message-ID: <8f2d0d19-c64f-d948-f81b-c6137382e64a@iem.at>

On 8/16/22 22:25, bil at ccrma.Stanford.EDU wrote:
> The error is hit in init_rootlet before the error_hook is set up,
> so it segfaults when it tries to call the error_hook.? The line
> number given in init_roolet is bogus, so I can't tell where it is.
> Actually a lot of stuff is messed up.? I guess we'll just have
> to disappoint the IBM mainframe guys.

so i ran the same test on another BigEndian architecture "ppc64", which 
is not run by default on our CI.

it has the same issue (and both backtrace and mus-config.h are the same)

after that, i compiled snd for PowerPC (32bit, LittleEndian), and 
*again* i got the same backtrace.
the mus-config.h is practically the same, except for SIZEOF_VOID_P which 
now reads "4" (as is expected for a 32bit system).

the machine i tested on had a Power8 CPU (for both tests).

to conclude: it seems that 'snd' is broken on BigEndian.


>? Back in 1969 (was
> it really more than 50 years ago?) I wrote some code (I
> think it was Basic, maybe Fortran) on an IBM mainframe
> (probably a 360?) -- I vaguely remember the typewriter
> (was it a typewriter?) had round keys??

sweet.

gmasdr
IOhannes

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://cm-mail.stanford.edu/pipermail/cmdist/attachments/20220817/98acdc70/attachment.sig>

From zmoelnig at iem.at  Wed Aug 17 01:09:44 2022
From: zmoelnig at iem.at (IOhannes m zmoelnig)
Date: Wed, 17 Aug 2022 10:09:44 +0200
Subject: [CM] snd on s390x (probably a BigEndian issue)
In-Reply-To: <8f2d0d19-c64f-d948-f81b-c6137382e64a@iem.at>
References: <fbace2ad46c09c11fdc887718fadb1ba@ccrma.stanford.edu>
	<8f2d0d19-c64f-d948-f81b-c6137382e64a@iem.at>
Message-ID: <35685503-ca37-5117-4644-31afcede9661@iem.at>

On 8/17/22 10:01, IOhannes m zmoelnig wrote:
> 
> to conclude: it seems that 'snd' is broken on BigEndian.
> 

on re-reading, this probably sounds harsher than i meant it to.

so to clarify: i'm not complaining (as i don't actually *use* Big Endian 
machines), merely stating a fact (afaict).

mfgsdrt
IOhannes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://cm-mail.stanford.edu/pipermail/cmdist/attachments/20220817/b93f5b37/attachment.sig>

From dev at mobileink.com  Fri Aug 26 06:04:57 2022
From: dev at mobileink.com (Gregg Reynolds)
Date: Fri, 26 Aug 2022 08:04:57 -0500
Subject: [CM] fn defined in C returns to wrong place
Message-ID: <CAO40Mi=wBOv7YsW2uFBCa8394xi4DY_OJWGxw3JaPzY6F4ONBg@mail.gmail.com>

I'm using s7 to convert from Dune (the standard OCaml build tool) to Bazel.
Dune files use sexp syntax, so in general this works great; almost all of
the conversion logic is written in s7 scheme.

But I've come across a problem that has me stumped.  The one
incompatibility I've found in dune syntax is use of a dot '.' to mean
"current directory". For example:

(run ocaml-protoc %{deps} -binary -pp -ml_out .)

Notice the dot at the end.  The s7 reader throws an error "unexpected close
paren: ..".  So I catch the error, close the input port, use the C api to
read the file into a C string (dune files are short), convert '.' to './',
and then s7 read the string from a string-port, giving a scheme list.
Works great.

The problem arises when I try to return the result. The routine that reads
the dune files is load-dune, written in C (I use the C fts api to crawl the
source tree, read the dune files, and construct a scheme object that
mirrors the source tree).  So my main routine, in C, loads my convert.scm
and then does s7_call (I've also tried s7_apply_function, no difference) to
run the main routine, which then calls load-dune.  The latter ends by
returning the scheme structure that results from crawling the source tree
and reading the dune files (using C apis).

This works great when the dune files can be read by s7 without error.
Control resumes in convert.scm, which then runs the remaining conversion
code (in scheme).  But when I encounter the bad-dot syntax, I handle the
error as described above, but when I 'return', control does not resume in
convert.scm.  Instead it resumes in the C code at the s7_call invocation
site. So the value I return is returned as the result of s7_call (or
s7_apply_function) which means the rest of my conversion code is skipped.

In other words it seems to skip the scheme call stack.  But I print the
call stack (using backtrace and backtrace_symbols), and its the same for
both good dune files and bad (corrected) dune files.  I also print a bunch
of debug msgs to stdout, and they are the same in both cases. The only
difference I can see is the return location.

I tried using s7_gc_protect functions in various ways to no effect.  I
tried disabling GC by s7_gc_on(s7, s7_f(s7)); but that had no effect.  I
tried eliminating the code that reads the file into a string by using a
string constant, to no effect.

So I think it must be related to the way I've configured error handling.
I've tried to follow the examples, e.g.

    old_err_port = s7_set_current_error_port(s7, s7_open_output_string(s7));
    if (old_err_port != s7_nil(s7)) {
        gc_loc = s7_gc_protect(s7, old_err_port);
    }

I also do this:

s7_pointer _s7_error_handler(s7_scheme *sc, s7_pointer args)
{
    if (strstr(s7_string(s7_car(args)), "unexpected close paren:") != NULL)
{
        if (debug)
            printf(RED "Error: BAD DOT" CRESET "\n");
        s7_write(s7, s7_make_string(s7, "BADDOT"),
                 s7_current_error_port(s7));
        return s7_t(s7);
    } else {
      ...
    }
   }

   s7_define_function(s7, "error-handler",
                       _s7_error_handler, 1, 0, false,
                       "our error handler");

    s7_eval_c_string(s7, "(set! (hook-functions *error-hook*) \n\
                            (list (lambda (hook) \n\
                                    (error-handler \n\
                                      (apply format #f (hook 'data))) \n\
                                    (set! (hook 'result) 'our-error))))");

and it all seems to work, insofar as the read error is caught, I detect
"BADDOT", and fix the read.  But the return does not work, and I have no
idea why.

Suggestions?

Thanks,

Gregg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cm-mail.stanford.edu/pipermail/cmdist/attachments/20220826/e1f3eec9/attachment.html>

From bil at ccrma.Stanford.EDU  Fri Aug 26 07:28:38 2022
From: bil at ccrma.Stanford.EDU (bil at ccrma.Stanford.EDU)
Date: Fri, 26 Aug 2022 07:28:38 -0700
Subject: [CM] fn defined in C returns to wrong place
In-Reply-To: <CAO40Mi=wBOv7YsW2uFBCa8394xi4DY_OJWGxw3JaPzY6F4ONBg@mail.gmail.com>
References: <CAO40Mi=wBOv7YsW2uFBCa8394xi4DY_OJWGxw3JaPzY6F4ONBg@mail.gmail.com>
Message-ID: <ae8af281d24fa7db9f6bf5a4d994cbf0@ccrma.stanford.edu>

*error-hook* is only called if there is no catch; it resets the s7 stack
when it returns, so you're back at the top level.  My first thought is
that you need a catch in convert.scm where you do the s7_call.  Maybe
I have the wrong model of what's happening.

The s7 stack can be viewed via (*s7* 'stack); it is independent of the
C stack (I assume that's what you're seeing with backtrace).  I don't
think the GC is the culprit.  s7_show_stack can give an abbreviated
view of the s7 stack from C (you'll need a declaration -- it's not
in s7.h, but it is accessible globally -- I thought of it as a
debugging aid -- (*s7* 'stack) is sometimes very verbose).


From dev at mobileink.com  Fri Aug 26 08:00:16 2022
From: dev at mobileink.com (Gregg Reynolds)
Date: Fri, 26 Aug 2022 10:00:16 -0500
Subject: [CM] fn defined in C returns to wrong place
In-Reply-To: <ae8af281d24fa7db9f6bf5a4d994cbf0@ccrma.stanford.edu>
References: <CAO40Mi=wBOv7YsW2uFBCa8394xi4DY_OJWGxw3JaPzY6F4ONBg@mail.gmail.com>
	<ae8af281d24fa7db9f6bf5a4d994cbf0@ccrma.stanford.edu>
Message-ID: <CAO40MikME3TdZTxMnG5rbqr7ZNnSEDVKYg3Lwv=CzfKz0F09vw@mail.gmail.com>

On Fri, Aug 26, 2022 at 9:28 AM <bil at ccrma.stanford.edu> wrote:

> *error-hook* is only called if there is no catch; it resets the s7 stack
> when it returns, so you're back at the top level.  My first thought is
> that you need a catch in convert.scm where you do the s7_call.  Maybe
> I have the wrong model of what's happening.
>
> The s7 stack can be viewed via (*s7* 'stack); it is independent of the
> C stack (I assume that's what you're seeing with backtrace).  I don't
> think the GC is the culprit.  s7_show_stack can give an abbreviated
> view of the s7 stack from C (you'll need a declaration -- it's not
> in s7.h, but it is accessible globally -- I thought of it as a
> debugging aid -- (*s7* 'stack) is sometimes very verbose).
>
> Thanks, that should be enough for further debugging.  I had completely
overlooked the s7 stack as distinct from the C stack. Pretty  sure at this
point it's a matter of getting my error management house in order. Which I
needed to to anyway; like a lot of stuff there's a (large) element of
seat-of-the-pants, hack it till it seems to work and move on. Actually
understanding How Things Work is sometimes useful, I find. ;)

Gregg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cm-mail.stanford.edu/pipermail/cmdist/attachments/20220826/377c4bec/attachment.html>

From dev at mobileink.com  Fri Aug 26 09:31:57 2022
From: dev at mobileink.com (Gregg Reynolds)
Date: Fri, 26 Aug 2022 11:31:57 -0500
Subject: [CM] fn defined in C returns to wrong place
In-Reply-To: <ae8af281d24fa7db9f6bf5a4d994cbf0@ccrma.stanford.edu>
References: <CAO40Mi=wBOv7YsW2uFBCa8394xi4DY_OJWGxw3JaPzY6F4ONBg@mail.gmail.com>
	<ae8af281d24fa7db9f6bf5a4d994cbf0@ccrma.stanford.edu>
Message-ID: <CAO40MikYYO5F71BR64FhhhjTKxy=oECKKi352qCLOyYuVaK+ew@mail.gmail.com>

On Fri, Aug 26, 2022 at 9:28 AM <bil at ccrma.stanford.edu> wrote:

> *error-hook* is only called if there is no catch; it resets the s7 stack
> when it returns, so you're back at the top level.  My first thought is
> that you need a catch in convert.scm where you do the s7_call.  Maybe
> I have the wrong model of what's happening.
>
> The s7 stack can be viewed via (*s7* 'stack); it is independent of the
> C stack (I assume that's what you're seeing with backtrace).  I don't
> think the GC is the culprit.  s7_show_stack can give an abbreviated
> view of the s7 stack from C (you'll need a declaration -- it's not
> in s7.h, but it is accessible globally -- I thought of it as a
> debugging aid -- (*s7* 'stack) is sometimes very verbose).
>

Looks like the s7 stack is the issue. I got rid of the error-hook.  Now my
'return' statement hangs.  Here is the code, details omitted:

for each dune file (composed of "stanza" sexps):

    while(true) {
        s7_pointer stanza = s7_read(s7, port);
        s7_show_stack(s7);
        errmsg = s7_get_output_string(s7, s7_current_error_port(s7));
        if ((errmsg) && (*errmsg)) {
            if (strstr(errmsg,
                       ";read-error (\"unexpected close paren:") != NULL) {
               ... fix problem, reading entire dune file ...
               break;
            }
        }
        if (stanza == s7_eof_object(s7)) {
            break;
        }
        ... do stuff with stanza ...
    }

If the file has no errors, the stack looks like this:

stack:
  let*1
  let*1
  eval_done
  gc_protect

But when we try to s7_read a stanza with the dot syntax error, the stack is:

reading stanza
stack:
  read_list
  read_list
  eval_done
  barrier
  let*1
  let*1
  eval_done
  gc_protect

I can still fix the error by reading the file into a string, fixing it, and
then s7_read the string-port. But now my return hangs.

Is there any way to unwind the stack past "barrier"?  Or is there a way to
call s7_read with a catch that will allow me to recover in situ?

Thanks,

Gregg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cm-mail.stanford.edu/pipermail/cmdist/attachments/20220826/19b66e6e/attachment-0001.html>

From bil at ccrma.Stanford.EDU  Fri Aug 26 10:25:55 2022
From: bil at ccrma.Stanford.EDU (bil at ccrma.Stanford.EDU)
Date: Fri, 26 Aug 2022 10:25:55 -0700
Subject: [CM] fn defined in C returns to wrong place
In-Reply-To: <CAO40MikYYO5F71BR64FhhhjTKxy=oECKKi352qCLOyYuVaK+ew@mail.gmail.com>
References: <CAO40Mi=wBOv7YsW2uFBCa8394xi4DY_OJWGxw3JaPzY6F4ONBg@mail.gmail.com>
	<ae8af281d24fa7db9f6bf5a4d994cbf0@ccrma.stanford.edu>
	<CAO40MikYYO5F71BR64FhhhjTKxy=oECKKi352qCLOyYuVaK+ew@mail.gmail.com>
Message-ID: <727a3296816e22b86edb8e82923977ad@ccrma.stanford.edu>

I think where you call s7_read, you want to wrap that
in a catch, probably using s7_call_with_catch.  There
are examples of it in ffitest.c.  The only tricky part
(I hope) is passing the "port" argument into the
thing called by the catch -- the catch "body" function
takes no arguments, as in scheme.