Physics Puzzle: Rope Between Two Poles

Posted on December 10, 2007
Filed Under Physics Puzzles | 1 Comment

A good puzzle, someone once told me, is one that can be solved by a bright high-schooler, and yet challenge even a seasoned practitioner. As a puzzle-aficionado, I second that notion, which is why mechanics puzzles are my favorites. It is the most intuitive discipline and the first to be studied in both high school and university, and yet the complexity of the phenomena it describes can often be quite astounding. Here is a puzzle to test your knowledge of mechanics:

A rope hangs between two poles. Can you calculate the tension in the rope at (a.) the lowest point, and (b.) at the points at which it connects to the poles? For the purposes of the question, assume the rope has some mass, m, which is uniformly distributed. Take L to be its length. You may assume you know the distance between the poles.

Note: although my graphical skills don’t necessarily portray this, the problem is symmetrical (i.e. the right and left poles are the same, and the rope is attached to both at the same height … )

Neural Networks Tutorial, Part #2

Posted on December 7, 2007
Filed Under Algorithms | 1 Comment

A Short Review

In the previous part of our tutorial, I’ve spoken about feed forward neural networks (FFNN), and said that the basic building block of a FFNN is a layer, which can be concisely summarized using the following diagram:

A layer has N inputs and M outputs (schematically represented here by a single line at the input and a single line at the output), and is characterized by three parameters: a weight matrix W, a bias vector b, and a transfer function, also called an input function, f. If the input of such a layer is given by the vector x = (x1, x2, …, xN), then the output, y=(y1,y2,…,yM) will be given by:

y = f(Wx + b)

There are a few input functions that are used repeatedly throughout the literature. Here are the most common 3:

The linear transfer function: $f(x) = \beta x$ , where beta is some constant.
The sigmoid hyperbolic tangent: $f(x) = \tanh (\beta x)$ , where beta is some constant.
The sigmoid function $f(x) = \frac{1}{1+ e^{-x/ \beta}}$ , where beta is some constant.

Let’s Get Practical

We will be studying a two-layer FFNN. Our goal will be to “teach it” the function f(x) = x^2 . We will do this by training the network on a set of inputs and outputs, and then hope it will manage to infer the function at all remaining points. Ok, beating around the bush. It’s time to introduce some notations. Here is a schematic representation of our network:

I’ve introduced a few notations in this diagram. Let’s go over them.

x stands for our input vector - at the moment we won’t any assumptions on its size, or the number of inputs/outputs of our network, although eventually we’ll choose our network to have a single input and a single output (the function has a single input, x, and a single output, f(x)).
. Be aware I’m using matrix notation here.
.
$\bar{h} = \bar{W} s + \bar{b} = \bar{W} f(h) + \bar{b}$ .
And finally, $\bar{s} = \bar{f} ( \bar{h} )$ is our output.

Note that $f, \bar{f}, W, \bar{W}, b, \bar{b}$ are all distinct variables. Although we will eventually choose f to be a sigmoid and $\bar{f}$ to be linear, we won’t need these assumptions to derive our conclusions below.

Training Our Network

As noted earlier,our final goal will be to train our network to mimic the function f(x) = x^2 . This means that we will “feed” our network with various inputs - say, x=1, 2, 3, 4, 5, etc. - and observe the outputs. Ideally, when inputting x, we’d like to see the network output its square - e.g., when inputting 7, we’d like to get 49 as our output. Obviously, this will not happen by default. We will need to observe the actual outputs, and adjust $W, b, \bar{W}, \bar{b}$ appropriately, until the actual outputs approximate the desired outputs closely enough. This is known as training the network.
How do we go about training our network? Fortunately for us, there is a nifty algorithm known as backpropagation which was invented for training FFNN. We will cover backpropagation in our next installment, so stay tuned!

MATLAB Speed: Using the M-Lint Utility

Posted on December 4, 2007
Filed Under matlab | Leave a Comment

A great deal of my work is done in MATLAB these days, and I come across many neat little features of the language and the IDE that are hidden among the menus. Sure, you’d know all about them if you read the manual, but who has time for that?

Here is a little trick to help you optimize your code. MATLAB comes with a utility called M-Lint that analyzes your code and tries to help you make it run faster. To enable it, in the main IDE, select the Desktop menu, -> Current Directory, to make the directory bar appear. This allows you to browse your projects and files easily:

Once you’ve enabled the directory view, scroll to the directory containing your project (your m-files). Next, select the M-Lint report generator from a tab that’s practically hidden in the directory view (fortunately for you I’m pointing it out):

Try finding that on your own! Once you select it, it automatically processes all of the m-files in the current directory and generates a report on each one:

The report is quite detailed, specifying line numbers where your code might be improved and suggesting how (and why!) it should be altered. This can even teach you quite a few things you’re not even aware of!

The Pendulum and Truck Puzzle

Posted on December 2, 2007
Filed Under Physics Puzzles | 3 Comments

Still working on the next part of our neural networks tutorial, but in the meantime, a puzzle.

A stationary truck has a pendulum of length L and mass m attached to the top of its insides (see figure A below). The truck driver hits the acceleration and as a result everything sort of gets “thrown back” inside the track - pendulum included (see figure B below). The question is, can the calculate the maximal angle the pendulum reaches as it is thrown back? Neglect air friction, or any other type of friction in this problem.

Enjoy!

Bus Stops and the Electron Charge

Posted on November 29, 2007
Filed Under Physics | Leave a Comment

While I’m preparing the next part in our neural networks tutorial, let’s digress a bit and talk about something completely different.

You’re standing outside, waiting for the bus to arrive. Being a physicist, you busy yourself with building some model until it does. Assume the rate of bus arrivals is $\lambda$ - for example, 5 buses per hour - and that buses are “memoryless” - that is, you can start your calculations and observations from the moment you’ve arrived at the bus stop without caring about what happened prior to your arrival, or what the hour was when you arrived. Since there are $\lambda$ buses per time unit, the probability that a bus will stop in the next few seconds, dt, is proportional to $\lambda dt$ . Let P(T) be the probability to not see a bus after time T. To derive an expression for P(T), we “chop up” the interval into N subintervals of equal length, T/N:

Assuming T/N to be sufficiently short, $\lambda T/N$ is the probability to see a bus in that time interval and $1-\lambda T/N$ is the probability to not see one. The probability to not see a single bus in the interval P(T) is simply the product of individual probabilities:

$P(T) \approx \left(1-\frac{\lambda T}{N} \right)^N$

Taking the limit $N \rightarrow \infty$ , and recalling that

$e^x = \lim_{N \rightarrow \infty} \left(1+\frac{x}{N} \right)^N$

we obtain

$P(T) = e^{-\lambda T}$

Puzzle: where did we make use of the “memoryless” property?
The next question we’ll ask ourselves is what is the number of buses we expect to see in a time interval $\Delta T$ . Let’s denote the probability of seeing n buses in that time interval by $P_n (\Delta T)$ . I’m going to ask you to take it on faith for now that that this is given by:

$P_n(\Delta T) = \frac{ (\lambda \Delta T)^n e^{-\lambda \Delta T}}{n!}$

This probability distribution is called a poisson distributions and events that are memoryless, such as bus arrivals (well, in theory at least!), are called poisson processes. It’s convenient to remember that a memoryless process with a rate lambda will be described by a poisson process, as derived above. Poisson processes are particularly easy to analyze because there is only one constant, $\lambda$ , which characterizes them. For example, the average number of events in the time interval $\Delta T$ will be:

$E[P_n(\Delta T)] = \lambda \Delta T$

and the variance will be:

$Var[P_n(\Delta T)] = \lambda \Delta T$

exactly the same! This is a unique property of the poisson process.

There are plenty of natural Poisson processes in nature. Here is an ingenious example of how to use them to measure a fundamental constant of nature, the electron charge: consider the weak current of electrons in a wire. Let’s assume we have a small number of electrons, N, such that the rate of electrons passing through a particular point is $\lambda$ , and let’s assume their flux is governed by a memoryless process, so the number of electrons we expect to see within a time interval $\Delta T$ will be governed by the poisson distribution. The mean number will be $\lambda \Delta T$ and the variance will equally be $\lambda \Delta T$ . Now, when you think about it, the current we measure will be given simply by the charge per electron times the number of electrons which pass a given point in a given unit time interval:

$I = e \dot{n}$

where $\dot{n} \equiv \frac{dn}{dt}$ is the number of electrons passing the point per unit time. Its mean will be:

$\bar{I} = e \bar{\dot{n}}$

If we assume the rate of passage of electrons is modeled by a Poisson process (since it’s electrons per unit time we’ll take $\Delta T = 1$ ) characterized by a rate $\lambda$ (where $\lambda$ has units of electrons/unit time), then:

$\bar{I} = e \lambda$

The current’s variance will be:

$\sigma_I^2 \equiv \bar{I^2} - \bar{I}^2 = e^2 \left[ \bar{\dot{n}^2} - \bar{\dot{n}}^2 \right] = e^2 \lambda = e \bar{I}$

where we have made use of the fact that the rate $\dot{n}$ is a poisson process, and hence its variance is equal to its mean. Our result tells us that, if we measure the current’s variance and divide it by its mean, the result will be equal to the electron charge:

$\frac{ \sigma_I^2 } { \bar{I}} = e$

The following figure gives a visual representation of the mean and variance of the current:

For this trick to work, though, one must be able to measure the variance due to the Poisson nature of the current. As it so happens, there are other sources of noise in a given system - such as thermal noise - which dominate the picture except for low temperatures and weak enough currents. Needless to say, this method of measuring e is only meaningful when all other sources of noise have been eliminated or sufficiently suppressed.

Neural Networks Tutorial, Part #1

Posted on November 25, 2007
Filed Under Algorithms, Signal Processing | 4 Comments

This is the first part of a set of postings on neural networks (NNs). NN are constructs that attempt to mimic our brain’s activity to a degree. NNs have become so common that it’s useful to know what people are talking about, so I’ve decided to write a brief tutorial - in several parts - on creating a simple NN. I’ll be providing code in MATLAB, but you should be able to implement what I say in any language.

What is a Neural Network?

The basic building block of a NN is a neuron. Each neuron has some value - a number - attached to it. Some neurons are binary, meaning they can only have a value of 0 or 1. Others can have any real value - those are the neurons we’ll be working with. Each neuron also has several incoming synapses and several outgoing synapses. The neurons are all interconnect, with the outputs of each wired into the inputs of others. When a neuron receives an input it processes the input somehow and “computes” the matching output. The way this computation is set up is completely up to the maker of the net. However, there are several computational models of networks that have proven themselves over the past few decades. We will be studying one such network, the Feed Forward Network.

What NNs Are There?

NNs are roughly divided into two parts: those that learn by themselves and those that are taught. Those that learn by themselves try to find regularity in a given set of data - for example, a NN might be given a large random set of internet sites and attempt to cluster them into categories based on which sites link where. Those that are taught are first trained on several pairs of inputs and outputs and, once the training it over, are supposed to classify new input appropriately - for example, a NN might initially be given the various letters of the alphabet in some person’s handwriting and told that this and that drawing corresponds to this and that letter in the alphabet. It can then be asked to decipher that person’s handwriting by identifying the letters used.

Our project will revolve around a small NN of the second kind that will “learn” the function y=x^2 . The net will first be trained with several pairs of numbers of the form (input, output), with the output being the square of the input. For example: (1,1), (2,4), (3,9), (4,16), (5,25). After this initial “training” the net will (hopefully) act as a simple calculator, returning the square of any input we feed it. Admittedly, this isn’t a very fancy network, but it has all of the ingredients of a more complicated network, and it’s simple to study.

Feed-Forward NNs

As mentioned The best and most common way of setting up a NN that learns by example is to use a feed forward neural network, dubbed henceforth FFNN. A FFNN is made of “layers” of neurons. Let’s look at one such layer:

A layer has inputs, x_1 and x_2 . These initial inputs are then “fanned out”, resulting in values y_1,y_2,y_3 . Note that the number of these intermediate values does not necessarily equal the number of inputs.

$y_1 = W_{11} x_1 + W_{12} x_2 \\ y_2 = W_{21} x_1 + W_{22} x_2 \\ y_3 = W_{31} x_1 + W_{32} x_2$

This can be rewritten using matrix notation:

$\left( y_1 \\ y_2 \\ y_3 \right) = \left( \begin{array}{cc} W_{11} & W_{12} \\ W_{21} & W_{22} \\ W_{31} & W_{32} \end{array} \right) \left( x_1 \\ x_2 \right)$

This can be succinctly written as

y=Wx

where y and x are vectors and W is a matrix.
Once the input has been “fanned out”, each intermediate value y_j gets a number, b_j , added to it. This is called a bias. Let’s denote:

h_j=y_j + b_j

or, in matrix notation:

$\left( h_1 \\ h_2 \\ h_3 \right) = \left( \begin{array}{cc} W_{11} & W_{12} \\ W_{21} & W_{22} \\ W_{31} & W_{32} \end{array} \right) \left( x_1 \\ x_2 \right) + \left( b_1 \\ b_2 \\ b_3 \right)$

We can write this succinctly as:

h=Wx + b

Finally, each h_j goes through an input function. In theory, each h_j could be processed by a different function, but in almost all real life situations, each intermediate value h_j is processed by the same function f. The exact form of f varies from network to network, but here are some common ones:

Finally, following the input function, the outputs - which we shall call s_j - will be

s_j=f(h_j)

This is often written, using matrix notation, as:

$s=f \left( Wx + b \right)$

This comprises the basic layer in a FFNN. As you can see, there’s no magic involved, nor are there any deep conceptual pitfalls to avoid. A convenient way of visualizing a layer is as a “black box”, with N inputs and M outputs:

Each layer is characterized by a matrix, W, a bias-vector, b, and an input function, f.
A FFNN is nothing more than a series of layers strung together. One must, of course, match the number of outputs of a layer to the number of inputs of the layer following it:

What Next?

How does one FFNN differ from another? As all FFNN are simply made of layers, and each layer is characterized by a matrix W, a vector b and a function f, we can only change a FFNN by altering W, b and f for each layer. In practice, the input functions f are rarely, if ever, changed. W and b are altered. It is possible to prove that by changing W and b any continuous function can be approximated. The trick is to find a method for changing the weights and biases for each layer such that it will “adapt” to the training set we present it with. This trick is called backpropagation, and will be covered in the next part of our tutorial. Stay tuned!

Exercise

Until next time, try and implement a feedforward network in your language of choice. I’ll be giving MATLAB code in our next tutorial, but I assume many of you have their own favorite language. FFNNs are natural candidates for object-oriented programming, as they are made out of layers. A layer is characterized by W, b and f - make it possible for the user to initialize the layer with any input function he chooses out of these three: y=tanh(x), y=x and y=1/(1+e^(-x)), also illustrated above. A layer should accept an input vector and output a vector as well (not necessarily of the same length, of course).

The Ice, Water and Oil Riddle

Posted on November 22, 2007
Filed Under Physics Puzzles | 1 Comment

Well, this is not really a riddle but more of a fun experiment.

We all know ice floats on water. We also know oil floats on water. Now the question is, if we take a glass and fill it with 50% water and 50% oil, and afterwards drop in an ice cube, where will that ice cube end up? Try to guess the answer first, and then give it a go in your kitchen.

Photos - to be published soon!

Added 29/11/07: yikes, my camera dropped dead - photos will have to wait a bit :). Check this post again in a week or so.

Added 2/12/07: ok, here it is!

The Psychotic Train Riders Puzzle

Posted on November 20, 2007
Filed Under Physics Puzzles | 5 Comments

Here is a somewhat cruel physics puzzle which illustrates the idea of momentum conservation.

Consider a train full of passengers moving along on a frictionless rail at a constant speed, v. In order to increase its speed, the passengers decide to throw themselves off the back end of the train, making use of the principle of conservation of momentum - much like a rocket expels fuel to propel it forwards.

Putting aside the question of their mental sanity, you, as a physicist, must now advise them on the best method of action. Seeing that there is only a limited number of people on board, should they (in order to achieve the greatest increase in speed):

Hurl the people one at a time, or
Throw themselves all at once?

Make the assumptions that all passengers have the same mass and throw themselves off the rear end of the train with the same velocity (relative to the train) equal to, say, -vf.

Getting Shot Isn’t So Bad For You

Posted on November 18, 2007
Filed Under Physics | 4 Comments

In this post we’ll be doing some simple, yet illuminating calculations.

A typical baseball weights about 150 grams and can reach velocities of up to 100 mph, or 40 m/sec, and so its kinetic energy is about $K = \frac{1}{2} m v^2 \approx 120 J$ and its momentum about $p = mv \approx 6 kg \cdot m \cdot sec^{-1}$ . On the other hand, the bullets of a .38 special handgun - the weapon of choice of many police departments in the US - weight about 8 grams, and have a muzzle velocity of roughly 300 meters per second. The kinetic energy of such a bullet is about $K = \frac{1}{2} m v^2 \approx 350 J$ , and its momentum about $p = mv \approx 2.4 kg \cdot m \cdot sec^{-1}$ . The two seem to be characterized by similar numbers, and yet we somehow expect a person shot with a bullet to drop down like a rag, and a person struck with a baseball to simply say, “ouch!”

If you’re expecting me to come up with some sort of ingenious physical explanation of this difference, you are in for a surprise. There isn’t any.

Studies conducted by the FBI conclude that most people fall down after being shot because they think they should. Most of us have been conditioned since an early age by the media to react in a certain way to getting shot. The FBI study basically noted that people who were intoxicated or simply well trained and experienced did not fall down after being shot once, unless the shot happened to have hit a critical region in the body, such as the spinal cord - and that doesn’t happen often, as less than 25% of the bullets shot at close range hit anybody at all, and a small percentage of those are lethal.

The myth that one drops dead when shot is called the one shot myth. It’s a fundamental problem most law enforcement agencies have to deal with, as most dangerous criminals - drug addicts, raging psychos and violent alcoholics - can’t simply be stopped by being shot at with a simple pistol. A rifle would do the trick, but it’s difficult to carry around and to draw quickly.

Beating your Guitar into Tune

Posted on November 16, 2007
Filed Under Physics | 6 Comments

If you have any musical friends, this should get their attention (if it doesn’t, just spill petrol over their musical instruments and ignite. THAT will).

Basic trigonometry tells us that the rule for adding two sines is as follows:

$\sin(\omega_1 t) + \sin(\omega_2 t) = 2 \cos \left( \frac{(\omega_2 - \omega_1) t}{2} \right) \sin \left( \frac{(\omega_2 + \omega_1) t}{2} \right)$

It turns out that if our ear is exposed to two sounds which have similar frequencies, it only hears the term which contains the difference of the frequencies. The fast-oscillating term,

$\sin \left( \frac{(\omega_2 + \omega_1) t}{2} \right)$ ,

is “averaged” and goes unnoticed. This is illustrated in the figure below, where I have plotted (in blue) the sum of sin(4t) + sin(4.5t) for values of t between 0 and 40. The red envelope corresponds to the function

$2 \cos \left( \frac{(\omega_2 - \omega_1) t}{2} \right) = 2 \cos(0.25t)$ ,

which is what our ear would hear:

The behavior shown above - that of a fast signal modulated by a slower one - is called beating in physics. This very phenomenon is utilized by guitar players, albeit unknowingly in most cases, to tune their guitars. Let’s take a look at a guitar fretboard, which is the name guitarists give to the long “neck-part” of their guitars:

There are 6 strings on a guitar. Every string on the guitar, should we pluck it, resonates at a different frequency, as each string has a different girth. The frequencies are listed above, along with the corresponding notes (E2, A2, D3, G3, B3 and E4). By pressing down on the strings at different points, though, the guitar player can effectively shorten the strings’ lengths and vary their pitch - and hence produce melodies. This is the basic physics of all stringed instruments. As a string is shortened its pitch is raised, so it sounds “higher”.

As it so happens, guitars tend to go out of tune with time, so, for example, the 2nd string from the top (B3) might have a frequency different than 247 Hz (perhaps 240 Hz), due to slackening of the string (caused by rust and mechanical wear). Guitarists have devised a quick test that helps them tune their guitar using the phenomenon of beating described above.

Suppose, for the sake of concreteness, that we suspect the 2nd string has gone slightly out of tune, and suppose that we know somehow by divine intervention that the 1st string, E4, is perfectly in tune. We then press the 2nd string down at the 5th, shortening its length such that - if plucked - its pitch would be 330Hz, the same as that of the uppermost string, E4:

We can now make use of the beating phenomenon to see if the 2nd string is out of tune: simply hold it down as in the above figure and pluck it together with the 1st string. If they are in tune, the two strings will vibrate at the same frequency and we will hear, effectively, a single note:

$s(t) = \sin(2 \pi \times 330 \times t) + \sin(2 \pi \times 330 \times t) = 2\sin(2 \pi \times 330 \times t)$

However, if they are not, and the 2nd string is not perfectly in tune, its frequency will differ slightly from 330 Hertz - say, 325 Hertz - and we will hear the beating produced by the two strings:

$s(t) = \sin(2 \pi \times 330 \times t) + \sin(2 \pi \times 325 \times t) \\ = 2 \cos(2 \pi \times 2.5 \times t) \sin(2 \pi \times 655 t)$

The guitarist’s ear will only detect the $\cos(2 \pi \times 2.5 \times t)$ term. He will then retune the string - using special tuning pegs at the end of the neck - until the beating goes away. This will indicate that they are both in tune. This beating can even be felt if you hold the guitar’s body against yours and pay careful attention.

Beating can only be used in a comparative way. It can tell us if two strings are out of tune with respect to each other. We will still, however, need some sort of reference to tune the absolute pitch of one of the strings.

« go back — keep looking »

About

Tune in for weekly posts on physics, mathematics and general science.
Search