Recently, somewhere, I ran across references to Echo State Networks (ESN) and to Liquid State Machines (LSM). I'm not sure where the original pointer came from, but I will track it down eventually. (That first link is dead, try Echo State Networks at Scholarpedia, Fraunhofer doesn't believe.)
As the principals claim, these are essentially the same line of research: ESN is about engineering signal processing systems while LSM is about understanding natural nervous system structure. ESN works with "neurons" in the usual computer abstraction of summing weighted activations and thresholding through a sigmoid. LSM works with "neurons" in the usual biological abstraction of trains of impulses resulting in stimulation or inhibition of the firing of further impulses. But both are about how to use neural networks which are connected at random without getting too bent out of shape about the lack of detailed understanding of how the randomly connected network works.
The first thing to get out of the way is the idea that we're talking about "neural nets" in the sense that most computing literature has used the term for the past 20 years. Those nets lack the recurrent structures, the feedback loops, which make the ESN and LSM networks both interesting and intractable.
The nets used in ESN are generated by randomly assigning weights between the hidden units. The weights are made to be sparse; the weights are scaled to bring the maximum absolute eigenvalue of the weight matrix to a desirable magnitude; and the weights are scaled to bring the maximum singular value of the weight matrix to a desirable magnitude. But that's the extent of the shaping applied to the network.
Neural nets as we know them from 20 years of research are much different sorts of beasts. They are built of layers of units where there are connections between layers coming from the inputs to the outputs, but no connections going back in the other direction. Introducing connections back to earlier layers from later layers would introduce nonlinear feedback loops into the networks.
ESN takes the randomly generated network as a black box. It doesn't attempt to engineer the structure of the randomly generated network other than in terms of sparsity, maximum absolute eigenvalue, and maximum singular value. So there's no interest in the detailed topology of the random network.