Projects in vision and information processing

We undertake many research projects in vision and information processing. 

 

 

Hyperspectral Image Analysis

Hyperspectral images provide both spatial and spectral representations of scenes, materials, and sources of illumination. They differ from images obtained with a conventional RGB colour camera, which divides the light spectrum into broad overlapping red, green, and blue image slices that when combined seem realistic to the eye. By contrast, a hyperspectral camera effectively divides the spectrum into very many thin image slices, the actual number depending on the camera and application [1]. This fine-grained slicing reveals spectral structure that may not be evident to the eye or to an RGB camera but which does become apparent in a range of visual and optical phenomena encountered in viewing and imaging natural scenes, including metamerism [2] and colour constancy [3]. Sets of hyperspectral images of natural scenes and supporting documentation are available at [4] and [5]

[1] http://personalpages.manchester.ac.uk/staff/d.h.foster/Tutorial_HSI2RGB/Tutorial_HSI2RGB.html

[2] Foster, D.H., Amano, K., Nascimento, S.M.C., & Foster, M.J. (2006). Frequency of metamerism in natural scenes. Journal of the Optical Society of America A-Optics, Image Science, and Vision, 23 (10), 2359-2372.

[3] Foster, D.H. (2011). Color constancy. Vision Research, 51, 674-700. doi:10.1016/j.visres.2010.09.006.

[4] http://personalpages.manchester.ac.uk/staff/d.h.foster/Hyperspectral_images_of_natural_scenes_02.html

[5] http://personalpages.manchester.ac.uk/staff/d.h.foster/Hyperspectral_images_of_natural_scenes_04.html

Contact: D. H. Foster |  d.h.foster@manchester.ac.uk

Information-Theoretic Estimates of Surface-Colour Coding in Natural Scenes

Measures of uncertainty drawn from Shannon's information theory allow the relationship between images of a scene taken under different conditions to be compared in a nonparametric way. This technique has been used to compare digital-camera sensors and provide ranking data for commercial colour cameras [6]. It has a range of other applications including the design of colour cameras and matching camera sensors to specific imaging applications. The same methods have been used to determine limits on the information retrievable by observers from natural scenes [7].

[6] Marín-Franch, I., & Foster, D.H. (2013). Estimating information from image colors: An application to digital cameras and natural scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 78-91.

[7] Foster, D.H., Marín-Franch, I., Amano, K., & Nascimento, S.M.C. (2009). Approaching ideal observer efficiency in using color to retrieve information from natural scenes. Journal of the Optical Society of America A, 26 (11), B14-B24.

Contact: D. H. Foster | d.h.foster@manchester.ac.uk

Engineering approaches to biological vision

Measures of uncertainty drawn from Shannon's information theory allow the relationship between images of a scene taken under different conditions to be compared in a nonparametric way. This technique has been used to compare digital-camera sensors and provide ranking data for commercial colour cameras [6]. It has a range of other applications including the design of colour cameras and matching camera sensors to specific imaging applications. The same methods have been used to determine limits on the information retrievable by observers from natural scenes [7].

[6] Marín-Franch, I., & Foster, D.H. (2013). Estimating information from image colors: An application to digital cameras and natural scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 78-91.

[7] Foster, D.H., Marín-Franch, I., Amano, K., & Nascimento, S.M.C. (2009). Approaching ideal observer efficiency in using color to retrieve information from natural scenes. Journal of the Optical Society of America A, 26 (11), B14-B24.

Contact: D. H. Foster | d.h.foster@manchester.ac.uk

Mechanisms Underlying Visual Coding of Surface Colour in Natural Scenes

Opto-electronic imaging of the eye and real-time control of high-resolution image displays allows inferences to be made about visual processing in the human eye and brain. Recent work has been concerned with changes in vision in the eyes of individuals with defective colour vision [8] and with recording patterns of eye movements as observers search for targets in images of natural scenes [9]. Other related work is concerned with the informational properties of the light signals reflected from natural scenes [10] and the development of model-free statistical methods for analysing eye-movement data [11].

[8] Carroll, J., Baraas, R.C., Wagner-Schuman, M., Rha, J., Siebe, C.A., Sloan, C., Tait, D.M., Thompson, S., Morgan, J.I.W., Neitz, J., Williams, D.R., Foster, D.H., & Neitz, M. (2009). Cone photoreceptor mosaic disruption associated with Cys203Arg mutation in the M-cone opsin. Proceedings of the National Academy of Sciences, 106 (49), 20948-20953.

[9] Amano, K., Foster, D.H., Mould, M.S., & Oakley, J.P. (2012). Visual search in natural scenes explained by local color properties. Journal of the Optical Society of America A - Optics Image Science and Vision, 29, A194-A199.

[10] Feng, G., & Foster, D.H. (2012). Predicting frequency of metamerism in natural scenes by entropy of colors. Journal of the Optical Society of America A - Optics Image Science and Vision, 29, A200-A208.

[11] Mould, M.S., Foster, D.H., Amano, K., & Oakley, J.P. (2012). A simple nonparametric method for classifying eye fixations. Vision Research, 57, 18-25.

Contact: K. Amano or D. H. Foster | k.amano@manchester.ac.uk or d.h.foster@manchester.ac.uk

Estimating Stimulus-Response and Transducer Functions

The response of an organism to the strength of a stimulus (e.g. a flash of light or a burst of noise) is described by the so-called psychometric or transducer function, from which summary measures such as a threshold or slope may be derived. Traditionally, this function is estimated by fitting a parametric model to the experimental data, usually the proportion of successful trials at each stimulus level. Common models include the Gaussian, Weibull, and reverse Weibull cumulative distribution functions, as shown in Figure 1. This approach works well if the model is correct, but it can mislead if not. In practice, the correct model is rarely known.

Estimating Stimulus-Response and Transducer Functions  

Here, a non-parametric approach has been developed based on local linear fitting. Because no assumption is made about the shape of the true function underlying the experimental data, except its smoothness, fitting may be regarded as model-free for experimental purposes. A non-parametric fit is shown in Figure 2. Current work is concerned with developing Matlab and R implementations of the software for use by the community.

Examples of alpha-level software are available as Matlab files for download.

Contact: D. H. Foster |  d.h.foster@manchester.ac.uk

Independent Component Analysis and Blind Signal/Image/Video Deconvolution and Denoising

The main goal of the project was to explore the ICA principles and their learning methods in practical image/video deblurring and restoration. It is also intended to extend nonlinear signal decomposition and analysis. Initial investigation has shown promising results of using ICA principles in denosing [1], then a blind deconvolution and deblurring algorithm based on kurtosis or negentropy measures and learning algorithms was proposed [2,3]. The project further addressed practical applications. The project investigated degradations caused by out-of-focus, motion and atmospherical blur.

Further improvement was made by using spectral kurtosis [4] and other quality measures [5]. The algorithm become more efficient and can be implemented for real-time applications.

However, nonlinear mixture models can take into account such blurring effects as an element of an imperfect acquisition system. Appropriate developments will be exploited in association with our external collaborators to promote the application of emergent technologies to real-world tasks. Of particular interest are optimum restoration low-level intensified images and low bandwidth night and infra-red surveillance video.

Contact: H. Yin | hujun.yin@manchester.ac.uk

Mutual Information “Decoder” for Investigating Neuronal Responses

Neuronal responses are usually quantified by counting the numbers of spikes evoked by a set of stimuli during a time window or local field potentials. However neurons can transmit information about sensory stimuli through their firing rate, spike latency and spike patterns. Information theoretic methods are a major force in dissecting the spike trains and can provide a systematic and principled interpretation compared to simple statistic or frequency methods. Information theory, the mathematical theory of communications systems, conceptualises the nervous system as a “communication channel” transmitting information about the sensory environment. Shannon’s mutual information between stimuli s and neuronal responses r can be used to quantify how much information neuronal population responses carry about external stimuli and to determine objectively the “code” used by neural populations to transmit information. This project is to apply mutual information measures for decoding the relationships between stimuli and responses. The study involves development of algorithms on both simulated brain signals that match the first and second order statistics of real neural recordings and a set of real recordings.

The project has developed a method termed, Information-Preserving EMD (Empirical Mode Decomposition). It has been applied to various neuronal response data including Local Field Potentials [1] and fMRI [2]. The results demonstrate the usefulness and effectiveness of the method in extracting underlying oscillations contained in the recordings. 

[1] Z. Mehboob and H. Yin: Information Quantification of Empirical Mode Decomposition and Applications to Field Potentials. Int. J. Neural Syst. 21(1): 49-63 (2011).

[2] Z. Mehboob, H. Yin, S. M. Wuerger, and L. M. Parkes: Multivoxel Pattern Analysis Using Information-Preserving EMD. IDEAL 2012: 19-26.

Contact: H. Yin | hujun.yin@manchester.ac.uk

Dimensionality Reduction and Face Recognition

Face recognition and identification has been an active research topic in recent years, amid high security and identity demand. With the ever increasing popularity of digital cameras and media, large face databases become available and often over distributed site. It has also become more and more difficult to identify and retrieve a query face (often noisy, partial or occluded) from a large database with ever increasing databases, especially over mobile communications networks. Many face recognition techniques employ either the PCA based Eigenface method or the LDA based Fisherface method. While both methods can and have produced good performance in many reported static applications in the literature, they are essentially linear methods and often cannot cope with increased complexity, nonlinearity and different level of noise in the distributed data sets. Identifying face images of low resolutions and poor quality over a communication network poses a great challenge to the practical application of face recognition techniques and security systems.

This project will investigate adaptive neural network based approaches for both pre-processing, such as filtering and dimensionality reduction of the face images, and the classification and recognition tasks. Retrieval from distributed, inhomogeneous databases will also be investigated under this application. The techniques include clustering and classification methods, the self-organising maps, radial basis functions and support vector machines. In particular ViSOM, entropy-based SOM (SOMN), and neural gas will be studied and enhanced for applications of face recognition and retrieval in distributed, semisupervised and low resolution environment. Experiments will be conducted on both benchmarks data sets such as ORL, Yale and Ferret, and locally gathered facial images.

Contact: H. Yin | hujun.yin@manchester.ac.uk

Enhancement of Broadcast Quality Video in Poor Visibility Conditions

This project is concerned with development of digital image enhancement algorithms for processing high-definition broadcast video to correct for the effects of atmospheric scattering and uneven illumination. The group has developed advanced algorithms for atmospheric correction [1] [2] and has launched a spin-out company, Dmist Ltd, that offers a commercial product for image enhancement [3]. The challenge in this project is to improve the accuracy and computational efficiency for this demanding application. Currently, broadcast streams from sporting events are corrected by highly-skilled camera operators using manual controls on the camera unit itself. The aim is to provide an automatic system that improves on this procedure in terms of quality and operator workload. Even slight errors in the processing may be noticeable and the challenge in this project is to ensure reliability under a wide range of imaging conditions. The techniques are based on physical models for loss of contrast due to light scattering in conjunction with novel statistical analysis [4]. The work will involve algorithm design/analysis using MATLAB, real-time programming in C++ and collaboration with organisations involved in generating broadcast content.

  1. K. K. Tan and J. P. Oakley, J. Opt. Soc. Am. A, vol. 18, pp. 2460-2467, 2001.
  2. J. P. Oakley and B. L. Satherley, IEEE Trans. Image Process., vol. 7, pp. 167-179, 1998.
  3. Dmist Technologies Ltd
  4. J.P. Oakley and H. Bu., IEEE Trans Image Processing, Vol. 16, No. 2, pp 511-522, 2007

Contact: Dr John Oakley |  john.oakley@manchester.ac.uk

▲ Up to the top