In the following, it is a list of the topics and articles for the midterm assignment. You can express your preference for one of the topics: the decision on topic assignment is ultimately made by the course instructor.
Prepare a 10-minutes presentation answering the associated questions (maximum number of slides should be 5/6).
Reading Material: T. Voegtlin “Recursive self-organizing maps.” Neural Networks 15.8 (2002): 979-991. pdf
Questions: Describe the recursive encoding of sequences in the RSOM. Report and discuss the network error and the update equations for the network weights. Provide a comparison between RSOM, temporal SOM and recurrent SOM (also showing the differences in the respective activation functions).
Reading Material: D. Bacciu, P. Barsocchi, S. Chessa, C. Gallicchio, A. Micheli, An experimental characterization of reservoir computing in ambient assisted living applications, Neural Computing and Applications, vol. 24 (6), pag. 1451–1464, 2014 pdf
Questions: Describe the application and the experimental scenario: highlight the differences between the homogenous and heterogeneous settings. Describe the leaky integrator echo state network: discuss changes (also with equations) with respect to the standard ESN. Why is the leaky integrator needed?
Reading Material: Rodan, P. Tino, Minimum complexity echo state network, IEEE Transactions on Neural Networks, vol. 22(1), pag. 131-144, 2011 pdf
Questions: Describe the DLR, DLRB and SCR topologies of an ESN. Sketch the demonstration of the memory capacity MC for an SCR (theorem 1). Summarize the experimental results: what minimal topology/parameterization has performance levels comparable to standard ESNs?
Reading Material: Hochreiter, Sepp, and Jürgen Schmidhuber. “Long short-term memory.” Neural computation 9.8 (1997): 1735-1780. pdf
Questions: Explain the vanishing gradient problem. Describe the LSTM architecture and main equations. What is the role of the gate units?
Reading Material: D.Bacciu, T.A. Etchells, P.J.G. Lisboa and J. Whittaker, “Efficient identification of independence networks using mutual information”, Computational Statistics, Springer, vol 28, no. 2, pp 621-646, Apr. 2013 pdf
Questions: Summarize the standard PC algorithm: describe the test of conditional independence and how it is computed with Mutual Information. Explain what is a False Negative in this scenario and describe the idea of power correction for reducing false negatives. Describe the concept of strong and weak edges and how/why this is used for the test-the-weakest-first policy.
Reading Material: Section 8.3.3 from Bishop chapter (pdf).
Questions: Describe the problem and the associated Markov random field. Provide the energy function equations and discuss their interpretation for the particular application.
Reading Material: Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., & Soda, G. (2001). Bidirectional dynamics for protein secondary structure prediction. In Sequence Learning (pp. 80-104). Springer Berlin Heidelberg. pdf
Questions: Describe the bi-diretional IO-HMM and discuss the equation for its joint distribution factorization: identify the model parameters and what are the stationariety assumptions. Summarize how the transition functions can be implemented using MLP neural networks.
Reading Material: Section 5.2.1 of David Barber’s Book [BRML].
Questions: Describe what is the typical max-product inference problem: why is different from sum-product? Describe the variable elimination idea in max-product. Describe the max-product message passing using factor graphs.
Reading Material: Jia Li, Markov Chain Interpretation of Google Page Rank, Tech Report ( pdf). Integrate with David Barber’s Book [BRML], pages from 461 to 463.
Questions: Describe the Pagerank algorithm from a Markov Chain point of view. Define the concepts of stationary and equilibrium distribution and discuss their interpretation in terms of Pagerank.