[CM] working with large data sets

[Lynn Artas]

Hello list,


I’m working with sonifying a large number of data points using grace. The
values are recorded as a .csv by another application, which I then open in
excel. From here, I have tried copying and pasting (paste special as
unformatted) into word (grace doesn't do anything when I try to replace ^M
or ^p) so I was using word for removing ^p and replacing with a space. I
realize grace might be working, but not be as obvious as word that it is
working on it.


Here are a couple specific problems:


1. formatting is a pain/fatal! both word and grace take minutes to paste
the data into a document, and word takes a few minutes just to search and
replace the ^p. I’ve tried to leave the data as is in the grace document
(if it works, it’s pretty enough for me), but I often get a crash just
trying to paste the data into a document in grace. Or a crash upon loading
the second 24000 point stream of data. (I'll need to load ten of these
24,000 streams) When I leave the document to do something else while it
churns, I get the spinning beach ball of doom when I return to
grace....aaaaand, as usual, just had grace crash again while typing this
while awaiting my paste.


When I try with raw (unformatted) data, I've had it crash as I load
(evaluate) the second 24,000 stream.


2. after I use Word to take out the ^p and replace with space, when I look
at the file with more in terminal, every number has a ^M (carriage return
in DOS/Windows-?!) in front of it and word won’t let me search and replace
the ^M. (^M is not a valid special character for the Find What box
according to MS word).


Here are my questions:


1. Is there a limit on the amount of data points you can load into memory,
or is it just the formatting characters gumming up the works?


2. And most important–can anyone outline a better way to work with big data
streams in grace?



Thanks for any help,

~L.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cm-mail.stanford.edu/mailman/private/cmdist/attachments/20160911/2bbb8853/attachment.html>