[CM] working with large data sets

Lynn Artas artaslynn at gmail.com
Sun Sep 11 17:07:18 PDT 2016

Hello list,

I’m working with sonifying a large number of data points using grace. The
values are recorded as a .csv by another application, which I then open in
excel. From here, I have tried copying and pasting (paste special as
unformatted) into word (grace doesn't do anything when I try to replace ^M
or ^p) so I was using word for removing ^p and replacing with a space. I
realize grace might be working, but not be as obvious as word that it is
working on it.

Here are a couple specific problems:

1. formatting is a pain/fatal! both word and grace take minutes to paste
the data into a document, and word takes a few minutes just to search and
replace the ^p. I’ve tried to leave the data as is in the grace document
(if it works, it’s pretty enough for me), but I often get a crash just
trying to paste the data into a document in grace. Or a crash upon loading
the second 24000 point stream of data. (I'll need to load ten of these
24,000 streams) When I leave the document to do something else while it
churns, I get the spinning beach ball of doom when I return to
grace....aaaaand, as usual, just had grace crash again while typing this
while awaiting my paste.

When I try with raw (unformatted) data, I've had it crash as I load
(evaluate) the second 24,000 stream.

2. after I use Word to take out the ^p and replace with space, when I look
at the file with more in terminal, every number has a ^M (carriage return
in DOS/Windows-?!) in front of it and word won’t let me search and replace
the ^M. (^M is not a valid special character for the Find What box
according to MS word).

Here are my questions:

1. Is there a limit on the amount of data points you can load into memory,
or is it just the formatting characters gumming up the works?

2. And most important–can anyone outline a better way to work with big data
streams in grace?

Thanks for any help,

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cm-mail.stanford.edu/mailman/private/cmdist/attachments/20160911/2bbb8853/attachment.html>

More information about the Cmdist mailing list