Blah, blah, blah. Those of you who were up to date on your Fourier transform are now bored out of your mind --- and those of you who weren't are still wondering why I'm talking so much about it rather than about the pen.

So now, it happens! Data is sent from the pen to my laptop, running Linux, via vibrating air! Doesn't it just put you at the edge of your seat?

The good news: "Hello, World!" did get from my pen to my desktop. The bad news: it did so at something like 8bps --- that's 8 bits per second, mind you. So it took something like a quarter of a minute just to send those two words.

Are improvements possible? Absolutely. I can increase my sampling rate (currently 8kHz, could easily be 16). I can increase the number of different frequencies to increase the per-unit-bandwidth.

But what is currently killing me is (I believe) the period between tones; not the tones themselves. There are two dangerous things that happen during the transition. One is that the Fourier analysis for time-bins which contain a frequency change have two peaks; this makes classifying those bins far more challenging, even classifying them as worthless and ignoring them.

Second, and a little more worrying, is on the pen: the amount of time that passes between the current sound clip ending after I request that the next sound play, and the next sound actually playing. It is, uh ... not zero. And that of course makes the the whole in-between-tone period even HARDER to identify and cope with.

Short note. Hopefully have more time next week to discuss possible fixes for all these faults.

Labels: livescribe, pulse, sam

So we have a rough idea what the frequency domain looks like; now we need some way of actually transforming data in the time domain (raw from the microphone) into frequency domain data. We'll do that through the creative application of interference.

For this trick to work, we must remember that the average value of a sinusoid is zero --- half the wave is positive, half the wave is negative, and it all cancels out nicely.

The other thing we need to know is how sinusoids multiply. So imagine a sinusoid, plodding along; a nice, slow wave with a long wavelength. Now imagine a fast, short wave sinusoid right on top of it. For each point along the x-axis of our imaginary graph, multiply the value of the slow wave with the value of the fast wave; and take care --- I said multiply, not add.

If your imagination is in smooth working order, you might piece together that the result is a periodic, oscillating wave. If your imagination is exceptionally good, you might even be able to notice that this curve is made up of two summed sinusoidal components. Truth be told, my imagination didn't hold up to the strain --- now I've got to find a new one. I recommend this for when your function imagination breaks down.

(In the meantime, we can examine a little math and discover that the two components are, in fact, sine waves whose frequencies are the sum of our original two sine waves and their difference, respectively.)

But regardless of what the components are, they are sinusoidal --- and we already said that the average value of a sine is zero.

What happens, though, if we take two identical sine waves, and multiply them? Every place that one is positive, the other is positive as well; a positive times a positive is another positive... and every place one is negative, the other is as well; negative times negative is also positive.

All of a sudden the average value is no longer zero!

More exciting still, the trick still works even when we use a sum of multiple sinusoids for our first wave --- if the sinusoid we're multiplying by is exactly the same frequency as one of the components of our first wave, then the average value of the product is non-zero --- and that value is proportional to the amplitude of the component wave. We've found a way to pick a single sine wave out of a sum of them! Our transform! Right?

There's an insidious evil lurking behind this joy.

Notice what happens when our two waves fall ninety degrees out of phase, so that the peaks of one wave occur simultaneous to the zeros of the other. When we multiply, BAM. another simple sine wave. That's no help --- its average is still zero. As it turns out, the average value of the product of same-frequency sines is not simply proportional to the amplitude of the original wave; it's proportional to the amplitude and phase shift. Shoot.

There is one hope, however: cosine. Cosine's already ninety degrees out of phase with sine, right? So whenever multiplying by a sine wave results in a false negative, multiplying by a cosine will get a full-strength output.

And sure enough, in combination, multiplying by sine, separately multiplying by cosine, and then adding the two results, negates the phase-based part of the new wave. Loverly. Here, we've invented the full form of the Fourier Transform (for that is indeed what it is): The amplitude of a given frequency component of a wave is equal to the average of the sum of a unit cosine at the given frequency times the original wave times a unit sine at the given frequency times the original wave.

That's it --- easy as pie, right?


Well, no, actually. Nowadays, no one uses that kind of notation --- we transform everything into the complex domain before we even begin. Even worse, there are huge pitfalls as soon as you make the whole thing digital --- when you no longer have smooth curves but individual samples the whole mess gets much more involved, plus there's the issue of efficiency; the method we used is about as fast as a mole on a paddleboat. It can't even reach the pedals, poor thing.


The practical application of all this comes right back to the Livescribe pulse. You remember that, right? All this has been about that. What we have now is a way to determine which frequency dominates a given audio sample. By making the pen play individual, pre-chosen frequencies, we can send data from the pen's speaker, through the PC microphone, and then decode the sounds back into labeled frequencies. Peachy. Now all we need is a way to encode data as a list of frequencies, and we're all set!

That will be the topic of my next post, where, with luck, I will be able to declare success in sending data from the pen to my PC without the use of Livescribe's desktop software.

So stay tuned.

Labels: livescribe, modem, pulse, sam

One of the frustrating things about the exciting livescribe pulse is it's inability to communicate with linux. Even through virtualization of a windows machine, you'll be lucky to get a working system.

Another is the fact that, while programs build with the livescribe pre-release SDK can save files to the pen, they cannot send those files to the desktop machine --- the only way to interact with program data is through the pen.

Well, I say pooh to that: "Pooh!"

Since I was first able to program my livescribe pulse (in a rude rendition of 'hello, world'), I've been scheming on how to communicate with the pen through hackish means. While reverse engineering the livescribe communications protocol would have been the correct way to do it, I feared it was, and is, beyond my means and motivation. I concentrated instead on the other two ways the pen has of communicating: the screen and the speaker.

My original plan had been to use light sensors, an AVR, and a program that would flash the pen's OLED display in order to send data back to the desktop computer. While this is plausible, the narrow bandwidth of the light produced by the OLED display makes it difficult to sense --- and the communication would always be one way. Besides, special hardware would be necessary, and that would make the potential user base tiny: linux-using livescribe owners who have the knowledge and hardware to program microcontrollers.

No, there must be a better way; so we turn to the speaker. Most computer owners have a microphone on hand; and communicating via tones is a tried and true method. My path is clear: create a livescribe modem!

In order to recieve information, the computer must be able not just to record the sounds coming from the pen, but to analyze them. To write software that can do that, we must have a firm understanding of the frequency domain. (Prerequisites note: If you don't have a firm understanding of the concepts frequency, wavelength, and sinusoidal graphs, it's time for a trip to wikipedia.

The central nugget of the matter goes like this: any continuous, periodic function can be represented as an infinite sum of sines and cosines.

Give that a minute to sink in; any function that is continuous --- not differentiable, just continuous --- and periodic --- but the length of the period can be as long as you like --- will do. Square waves, sawtooths, Bart Simpson's hair; any arbitrary waveform can be represented by a simple sum of sines and cosines.

Wow.

You may be worried about that word 'infinite' up there. Luckily, it turns out that we can approximate our function with some finite number k sines and cosines; and that each time we increase k, the approximation becomes more accurate.

Now let's take a moment to remember what the time domain, looks like. Imagine a signal coming from your microphone. It is a long string of numbers, and each number indicates a magnitude of sound at a single point in time (which is to say, the difference in pressure on the front and rear surfaces of the microphone pickup). If we were to graph the sound of someone whistling by using the time as the independent variable, and magnitude as the dependent, we'd see a lovely sine wave --- the archetypal image of sound, in the time domain.

But there is another way of plotting the same sound. We said that any curve --- no matter how complex our time domain curve might be --- can be represented by a sum of sines and cosines, right?

And, hold on to your hats, we only need 2 numbers to define a sine wave: the frequency of the wave, and the amplitude. So we have lots of data points enumerated by two real numbers? That sounds like a graph. And indeed it is. If we graph frequency as the independent variable and amplitude as the dependent, we have arrived at a graph of our sound in the frequency domain.

For those of you still having a little trouble visualizing, here's what the sound of me whistling looks like in the time domain:




And here's what it looks like in the frequency domain:



See that big sharp spike there around 800Hz? That is the primary frequency of the whistle (772Hz: a G5 that's a little flat). All the other stuff is the shower, the refrigerator, the computer fan, the traffic outside, the wind, and so on; but remember we're measuring in decibels, so they are whole orders of magnitude less in amplitude than the whistle.

Most of you are probably yelling at the screen by now, asking me how on earth we're supposed to find these numbers. Where do we get this sum of sinusoids if all we're given is the time domain information? The empty-words answer is "The fourier transform." To understand that answer, we're going to have to go in a journey --- one that I'll leave to my next post.

Labels: livescribe, modem, pulse, sam