[CM] working with large data sets

[Taube, Heinrich K]

hi lynn the bridge from grace’s editor to s7 is intended as a code editor and simply isnt made for huge wads of data.  

I would use an external app like emacs to edit the data in a .scm file and then simply load that file i Grace (or more speciically, into s7, which grace uses.)   so the scheme file would look like
(define mydata ‘(1 2 3
 ….. 24,000
))

also with huge data its possible may run out of heap space. if thats the case and depending on what you are tring to do, you could switch to the CM2/CLM/Common Lisp. those packages work will and common lisp would probably handle the large data sets that you are using.

> On Sep 11, 2016, at 7:07 PM, Lynn Artas <artaslynn at gmail.com> wrote:
> 
> Hello list, 
> 
> I’m working with sonifying a large number of data points using grace. The values are recorded as a .csv by another application, which I then open in excel. From here, I have tried copying and pasting (paste special as unformatted) into word (grace doesn't do anything when I try to replace ^M or ^p) so I was using word for removing ^p and replacing with a space. I realize grace might be working, but not be as obvious as word that it is working on it. 
> 
> Here are a couple specific problems:
> 
> 1. formatting is a pain/fatal! both word and grace take minutes to paste the data into a document, and word takes a few minutes just to search and replace the ^p. I’ve tried to leave the data as is in the grace document (if it works, it’s pretty enough for me), but I often get a crash just trying to paste the data into a document in grace. Or a crash upon loading the second 24000 point stream of data. (I'll need to load ten of these 24,000 streams) When I leave the document to do something else while it churns, I get the spinning beach ball of doom when I return to grace....aaaaand, as usual, just had grace crash again while typing this while awaiting my paste. 
> 
> When I try with raw (unformatted) data, I've had it crash as I load (evaluate) the second 24,000 stream.
> 
> 2. after I use Word to take out the ^p and replace with space, when I look at the file with more in terminal, every number has a ^M (carriage return in DOS/Windows-?!) in front of it and word won’t let me search and replace the ^M. (^M is not a valid special character for the Find What box according to MS word). 
> 
> Here are my questions:
> 
> 1. Is there a limit on the amount of data points you can load into memory, or is it just the formatting characters gumming up the works? 
> 
> 2. And most important–can anyone outline a better way to work with big data streams in grace?
> 
> 
> Thanks for any help, 
> ~L.
> 
> 
> 
> _______________________________________________
> Cmdist mailing list
> Cmdist at ccrma.stanford.edu
> https://cm-mail.stanford.edu/mailman/listinfo/cmdist