Research Experience for Undergraduates (REU) in Computational Sensing and Medical Robotics- 2015 Projects List

Please select top 3 projects from the list below and list the project numbers in your application.

Project 1: Analysis of Cerebellar Shapes

Diseases such as multiple sclerosis, alcohol and solvent toxicity, Lyme disease, and encephalitis  can cause atrophy of the cerebellum which can affect movement, coordination, and speech.  Novel ways to statistically analyze the size and shapes of  cerebellar lobules will be explored using data from both normal subjects and those with genetic spinocerebellar ataxias.  Relating function to structure is a key goal and the potential to discover novel relationships is high.

Required Skills: Experience with matlab and linear algebra is required; experience in the  Java programming language is desirable.
Faculty advisor: Prof. Jerry Prince

 

Project 2: Image Processing Pipeline for Analysis of the Retina.

Optical coherence tomography is a noninvasive ocular imaging technology that uses near-infrared light to produce cross-sectional and three-dimensional images of the retina.  This project involves creative use and development of image processing algorithms for analysis of the retinal layers in both healthy and diseased subjects.   The participant will work with a team that includes other students and engineers as well as medical doctors in order to find new ways to characterize subtle differences in the thicknesses of retinal layers.

Required Skills: Basic knowledge of signal processing or computer vision is required; experience in the Java programming language is desirable.

Faculty advisor: Prof. Jerry Prince

 

Project 3: Multisensory Control in Locomotion

Where two or more independent sensors are fused to control movement — is a fundamental feature of robotic systems and organisms alike. We have a remarkably well suited model system for investigating multisensory control in nature. Specifically, a species of fish, the glass knifefish, readily swims back and forth to stay “hidden” inside a virtual reality system we’ve created. The fish can see with their eyes, but they also generate an electric field which they can sense with their skin, giving them another independent sensory modality. Our system allows us to move the visual and electrosensory scenes completely independently and monitor how well the animal is able to stay within the hiding place. We then create conflicts between these two sensory cues that would be like your inner ear suddenly telling you you were falling to the left, but your eyes telling you you were falling to the right. How would your brain reconcile those two conflicting data streams? Investigations into the way the brain solves this problem will lead to new insights into the neural control of movement and, just perhaps, may enable us to build more effective multisensory control strategies for robots.

Required Skills: Undergraduate training in linear algebra and differential equations. Basic knowledge of Matlab or another programming (C, C++, Python). No specific experimental background in biological systems is required, but a lack of fear of getting into the lab (with mentorship) and performing non-invasive behavior experiments on live animals (specifically, fish) is essential.

Faculty Advisor: Dr. Noah Cowan

Website: http://limbs.lcsr.jhu.edu

 

Project 4: How do mice get back to their homes after they find the cheese?

This is a project on understanding how mice represent space and remember locations in their brains.  In fact, this year’s Nobel Prize in Biology was awarded for research on this problem.  REU students will work on modeling the hippocampus part of the brain in hardware (such as FPGAs) and use it to guide a mobile robot.  The student will program MatLab, VHDL and C/C++ to realize the navigation algorithms of neuroSLAM.  At the end of the summer, the student will be able to demonstrate how mouse leaves its home, remembers places in the world, finds food and returns home, while avoiding cats.

Faculty Advisor: Dr. Ralph Etienne-Cummings

 

Project 5: Large deflection shape tracking for dexterous surgical robot

Dexterous surgical robots have been commonly used in minimally invasive surgery (MIS) for their high steerability and flexibility. Surgeries like osteolysis curettage need intermittent intra-operative x-ray image for the navigation of the robots. However, a real-time tracking requires large irradiation dose which does great harm to patients. Also, the kinematic model based prediction is not accurate enough for the robot control. Shape tracking technology using optical fibers with multi Fiber-Bragg-Grating(FBG) sensors give us opportunity to monitor the shape of robot in real-time and without radiation. In this project, a tiny shape sensor will be designed and assembled to a dexterous surgical robot, and the shape will be reconstructed from wavelengths of FBGs captured by the interrogator. It is promising for better control of dexterous surgical robots and will greatly enhance the safety and efficiency of MIS.

Required Skills: Signal Processing, Differential Geometry, Error Analysis, Matlab Programming

Faculty Mentor(s): Mehran Armand/ Iulian Iordachita

 

Project 6: Steady-Hand Eye Robot with Multi-Function Force Sensing Instrument for Retinal Microsurgery

Project Description: Vitreoretinal surgery is one of the most challenging microsurgery disciplines. Robotic assistants and smart surgical instruments have the potential to help the surgeons to overcome many significant physiological limitations. In this project, we want to design and conduct experiments to evaluate a novel multi-function force sensing instrument, develop control methods for the Steady-Hand Eye Robot that incorporate this multi-function force sensing instrument to provide safe surgical intervention and intuitive user interface.

Required Skills: Programming (Matlab, Python, C++), CAD (Pro/E or Solidworks), Prototyping, good analytical skills

Faculty Mentor(s): Iulian Iordachita

 

Project 7: Robustness properties of logistic networks

Project Description: Logistic networks such as transportation networks and production-distribution networks need to operate reliably and economically under a wide range of conditions. The proposed project aims at complementing our ongoing studies of robustness of such networks,  by investigating computationally tractable ways of verifying relevant properties, and by setting up numerical simulations of case studies under various models of uncertainty.

Required skills: Experience with MATLAB and/or C++ coding. Strong background in Linear Algebra.

Faculty Advisor: Dr. Danielle C. Tarraf

 

Project 8: Machine Learning methods for Measuring (Disease) Activity Using Smart Phones: Application to Parkinson’s

The goals of this project are to understand how smartphones can be used in everyday settings to monitor health in individuals with neurodegenerative disorders.

In Parkinson’s, for example, medications lose their efficacy, and symptoms often reappear before it’s time for another dose. Real-time inference of an individual’s

health status can allow for triggers that make individualized recommendations for initiating a clinic visit or increasing dose. This project will involve developing novel

methods for inferring changes in an individual’s health status from streaming time series data of symptoms measured via smartphones, and methods for

tailoring interventions to the patient. Our study has assembled the largest and most comprehensive database of smartphone based measurements collected to date on

individuals with Parkinson’s disease and is continuing to grow at the rate of several hundred Gbs/week.

Faculty Advisor: Suchi Saria (Machine Learning and Applied Statistics); Ray Dorsey (Neurology)

 

Project 9: Early Detection of Adverse Events

More than one in five patients suffer from hospital acquired infections. Data that are routinely collected in the hospital environment can be used to predict individual’s at risk for adverse clinical events in real-time. By identifying at-risk individuals early, clinicians can begin more aggressive therapies that are targeted to the specific event sooner and significantly decrease risk of death. In this project, we will develop computational methods to infer an individual’s risk for adverse events over time based on the individual’s clinical temporal streams. The emphasis will be on developing methods that scale to large data sets and can be implemented in real-time.

Faculty Advisor: Suchi Saria (Machine Learning and Applied Statistics)

 

Project 10 : Telerobotic System for Satellite Servicing

Description:  With some satellites entering their waning years, the space industry is facing the challenge of either replacing these expensive assets or to develop the technology to repair, refuel and service the existing fleet.  Our goal is to perform robotic on-orbit servicing under ground-based supervisory control of human operators to perform tasks in the presence of uncertainty and time delay of several seconds. We have developed an information-rich telemanipulation environment, based on the master console of a customized da Vinci surgical robot, together with a set of tools specifically designed for in-orbit manipulation and servicing of space hardware.  We have successfully demonstrated telerobotic removal of the insulating blanket flap that covers the spacecraft’s fuel access port, under software-imposed time delays of several seconds. We now wish to extend the models and tools to other telerobotic servicing operations.

Required Skills:  Ability to develop mathematical models of satellite servicing tasks, implement models in C/C++, and perform experiments to determine model parameters.

Faculty Advisors:  Peter Kazanzides, Louis Whitcomb

 

 Project 11: Hybrid Tracking System for Surgery

Description:  Tracking systems are often used to show the position of surgical instruments with respect to preoperative images of the patient.  Essentially, it is a Global Positioning System (GPS) for surgery, which shows the position of the instrument (“moving vehicle”) with respect to the image (“map”).  Different technologies, such as optical and electromagnetic tracking, are used, with different constraints and levels of accuracy. We have developed a custom hybrid tracking system that combines electromagnetic and inertial sensing, to attempt to achieve high accuracy, even in the presence of magnetic field disturbances. The project goals are to implement and test different sensor fusion methods to improve the robustness to environmental effects such as magnetic field disturbances.

Required Skills:  Ability to perform laboratory experiments and analyze results using Matlab. Programming experience in C/C++ would be helpful, but not required.

Faculty Advisor:  Peter Kazanzides

 

Project 12: Photoacoustic Ultrasound Guidance for Neurosurgery

Description:  Photoacoustic ultrasound refers to an ultrasound image that is formed by using a pulsed laser to excite selected tissue to create an acoustic wave that is detected by an ultrasound receiver array.  This project explores the use of photoacoustic ultrasound to detect blood vessels behind bone during skull base surgery.  This information can be used to update the registration of the preoperative plan and to prevent the cutter from damaging the blood vessels.  The project goal is to perform phantom experiments to develop a measurement strategy (i.e., where to aim the laser) and to quantify the accuracy improvement due to the photoacoustic measurements.

Required Skills: Ability to perform laboratory experiments and analyze results using Matlab. Experience with ultrasound and programming experience in C/C++ would be helpful, but not required.

Faculty Advisors:  Peter Kazanzides, Sungmin Kim, Muyinatu Lediju Bell

 

Project 13 :Real-Time Sensor Fusion for Surgical Navigation

Optical tracking systems are often used to show the position of surgical instruments with respect to preoperative images of the patient, such as CT or MRI.  While optical tracking provides high accuracy, it requires a continuous line-of-sight between the optical tracker and markers on the patient and surgical tools, which is burdensome during an operation. To reduce the burden, and to make the surgical navigation system usable in cases where there are partial occlusions, we have developed a sensor fusion technique with an inertial measurement unit (IMU) attached to the tracker. We have demonstrated this method on recorded data using Matlab. The project goals are to make this technique usable in real time, and to perform experimental evaluations.

Required Skills: Ability to perform laboratory experiments and analyze results using Matlab. Programming experience in C/C++

Faculty Advisors:  Peter Kazanzides, Sungmin Kim

 

Project 14: Power system modeling and simulation studies for renewable energy integration Concerns regarding global warming, the finite nature of conventional energy reserves, energy security and rising costs are driving the need to find more efficient and renewable energy sources and systems. Unfortunately, renewable power sources, such as solar cells or wind farms, differ significantly from conventional power plants and integrating them into the power grid poses a number of challenges.  For example, dealing with the uncertainty of sun and wind availability and the inherent intermittency of these resources will require changes to the current infrastructure and system components.  Thus, achieving the full potential of “smart” and clean power systems will require a combination of different generation schemes, storage, ancillary services and other energy assets as well as an understanding of how to best coordinate and distribute these resources. This project involves modeling, simulation and analysis of power systems with varying levels of renewable energy.  In particular, studies aimed at assessing the stability, efficiency and costs of these systems.    Current research in this area includes  evaluating the placement of different energy resources in order to simultaneously mitigate risks and minimize costs given different network topologies.  The summer project will involve simulation studies of various scenarios to test the existing theory and explore different mitigation strategies under different performance criteria.  These simulations will be used to develop a model to analyze trade-offs between important aspects of the problem such as cost and efficiency versus reliability, stability and environmental concerns. 
Faculty Advisor: Dennice Gayme

 

Project 15 : Robotic System and Smart Instruments for Microsurgery
Our laboratory has an ongoing research partnership with surgeons from the Johns Hopkins Ophthalmology Department to develop a technology and systems to significantly enhance the ability of surgeons to perform delicate microsurgical interventions in the retina.

As part of this project we are developing novel microsurgical instruments capable of directly sensing tool-to-tissue interaction forces, as well as tissue characteristics and properties within the retina.  These tools may be used “free hand” or with robotic devices within our laboratory capable of making motions with 2-5 micron precision.  Robots and sensors are interfaced to a stereo video microscope, and we are also developing robotic and visualization software to integrate all this information and assist the surgeon in delicate tasks such as peeling membranes and inserting needles into 100 micron blood vessels.

A major part of our effort is experimental validation of the effectiveness of this technology and system in actually helping eye surgeons to perform such tasks.  The undergraduate student would participate actively with the entire team and would undertake a specific project, the details of which would be tailored to the student’s specific technical abilities.  Typical projects might include:

• Assisting in data gathering and analysis of operating room video of live surgery. (This project will not require construction of any hardware)
• Computing a light exposure map of the retina from video images (requires computer vision and strong programming skills)
• Measuring the tool-to-tissue interaction forces encountered in micro-retinal surgery on cadaveric eyes, in order to obtain baseline data for future experiments
• Designing and carrying out an experimental study comparing the ability of a surgeon using our system to control tool-tissue interaction forces to the performance achieved with conventional instruments.
• Development and evaluation of novel sensory substitution schemes combining auditory and visual feedback
  Required Skills: The student should have significant skills in designing experiments and in data reduction using MATLAB or similar packages, together with additional engineering skills (e.g., in biomedical engineering, mechanical engineering, electrical engineering, or computer science) appropriate for the individual project.  Experience in actual hardware design and interfacing is a definite plus, as would be strong programming skills in C++ (depending on details of the project). Similarly, familiarity with computer vision methods and/or statistical analysis of experimental data will be valuable for some projects.
• At the end of the project, we anticipate publishing a paper with the REU student as a co-author.

Faculty Advisor: Prof. Russell Taylor

 

Project 16 : Software environment and virtual fixtures for medical robotics
Our laboratory has an active ongoing research program to develop open source software for medical robotics research.  This “Surgical Assistant Workstation (SAW)” environment includes modules for real time computer vision; video overlay graphics and visualization; software interfaces for “smart” surgical tools; software interfaces for imaging devices such as ultrasound systems, x-ray imaging devices, and video cameras; and interfaces for talking to multiple medical robots, including the Intuitive Surgical DaVinci robot, our microsurgical Steady Hand robots, the IBM/JHU LARS robot, the JHU/Columbia “Snake” robot, etc.  A brief overview for this project may be found at https://www.cisst.org/saw/Main_Page.  Students will contribute to this project by developing “use case” software modules and applications.  Typical examples might include: using a voice control interface to enhance human-machine cooperation with the DaVinci robot; developing enhanced interfaces between open source surgical navigation software and the DaVinci or other surgical robots; or developing telesurgical demonstration projects with our research robots.  However, the specific project will be defined in consultation with the student and our engineering team.

Required Skills: The student should have strong programming skills in C++.  Some experience in computer graphics may also be desirable.

Faculty Advisor: Prof. Russell Taylor, Dr. Peter Kazanzides

 

Project 17 : Flexible Robot Endoscope for Laryngeal Surgery
Our laboratory has developed a robotically manipulated flexible endoscope for laryngeal surgery.  This robot is specifically designed to be qualified for clinical use, and we expect that cadaver studies will be completed and the IRB approval process is ongoing, with first clinical use expected in the coming year.  Depending on the progress of the IRB approval process, there will be several possible projects, including:

• Engineering support for early clinical trials of the system. In this case, the student will assist the surgeons in setting up the system and using the system in surgery, will gather real time video and robot motion data using existing software tools developed as part of our open-source CISST/SAW environment, and reduce the data for academic publication. This may require extending some of the software interfaces to capture the specific data required by the surgeons.
• Development of image registration and overlay capabilities for the system. In this case, the student will adapt existing software and algorithms developed within our laboratory to reconstruct 3D models of the patient’s anatomy from captured video, combined with motion data captured from the robot, register the data to preoperative 3D models, and generate 3D visual displays to assist the surgeon. Note: This project is extremely ambitious, and it may be that the student will only complete the first phase during the summer. However, even this will be an impressive accomplishment. Students interested in this project should have strong programming skills and experience with computer vision methods.
• Any of these projects is likely to result in academic publication in journals or conferences.

Required Skills: The student should have a background in biomedical instrumentation and an interest in developing clinically usable instruments and devices for surgery.  Specific skills will depend on the project chosen. Experience in at least one of robotics, mechanical engineering, and C/C++ programming is important.  Similarly, experience in statistical methods for reducing experimental data would be desirable.

Faculty Advisor: Prof. Russell Taylor, Dr. Jeremy Richmon (Dr. Lee Akst, Otolaryngology)

 

Project 18: Instrumentation and steady-hand control for new robot for head-and-neck surgery
Our laboratory is developing a new robot for head-and-neck surgery.  Although the system may be used for “open” microsurgery, it is specifically designed for clinical applications in which long thin instruments are inserted into narrow cavities.  Examples include endoscopic sinus surgery, transphenoidal neurosurgery, laryngeal surgery, otologic surgery, and open microsurgery.  Although it can be teleoperated, our expectation is that we will use “steady hand” control, in which both the surgeon and the robot hold the surgical instrument. The robot senses forces exerted by the surgeon on the tool and moves to comply.  Since the motion is actually made by the robot, there is no hand tremor, the motion is very precise, and “virtual fixtures” may be implemented to enhance safety or otherwise improve the task.  Possible projects include:

• Development of “phantoms” (anatomic models) for evaluation of the robot in realistic surgical applications.
• User studies comparing surgeon performance with/without robotic assistance on suitable artificial phantoms.
• Optimization of steady-hand control and development of virtual fixtures for a specific surgical application
• Design of instrument adapters for the robot
  Faculty Advisor: Prof. Russell Taylor, Dr. Jeremy Richmon (Otolaryngology), Dr. Masaru Ishii (Otolaryngology), Dr. Lee Akst (Otolaryngology), Dr. Wade Chien (Otolaryngology)
• Required Skills: The student should have a background in biomedical instrumentation and an interest in developing clinically usable instruments and devices for surgery. Specific skills will depend on the project chosen. Experience in at least one of robotics, mechanical engineering, and C/C++ programming is important. Similarly, experience in statistical methods for reducing experimental data would be desirable.

 

Project 19: Software environments and virtual fixtures for medical robots
Our laboratory has an active ongoing research program to develop open source software for medical robotics research.  This “Surgical Assistant Workstation (SAW)” environment includes modules for real time computer vision; video overlay graphics and visualization; software interfaces for “smart” surgical tools; software interfaces for imaging devices such as ultrasound systems, x-ray imaging devices, and video cameras; and interfaces for talking to multiple medical robots, including the Intuitive Surgical DaVinci robot, our microsurgical Steady Hand robots, the IBM/JHU LARS robot, the JHU/Columbia “Snake” robot, etc.  A brief overview for this project may be found at https://www.cisst.org/saw/Main_Page.  Students will contribute to this project by developing “use case” software modules and applications.  Typical examples might include: using a voice control interface to enhance human-machine cooperation with the DaVinci robot; developing enhanced interfaces between open source surgical navigation software and the DaVinci or other surgical robots; or developing telesurgical demonstration projects with our research robots.  However, the specific project will be defined in consultation with the student and our engineering team.

Required Skills: The student should have strong programming skills in C++.  Some experience in computer graphics or computer vision may also be desirable.

Faculty Advisor: Prof. Russell Taylor, Dr. Peter Kazanzides

 

Project 20: Statistical Modeling of 3D Anatomy
Description: The goal of this project is creation of 3D statistical models of human anatomic variability from multiple CT and MRI scans.  The project will involve processing multiple images from the Johns Hopkins Hospital, co-registering them, and performing statistical analyses.  The resulting statistical models will be used in ongoing research on image segmentation and interventional imaging.  We anticipate that the results will lead to joint publications involving the REU student as a co-author.

Required skills: Experience in computer vision, medical imaging, and/or statistical methods is highly desirable.

Faculty advisor: Prof. Russell Taylor

 

Project 21: Accuracy Compensation for “Steady Hand” Cooperatively Controlled Robots
Description: Many of our surgical robots are cooperatively controlled.  In this form of robot control, both the robot and a human user (e.g., a surgeon) hold the tool.  A force sensor in the robot’s tool holder senses forces exerted by the human on the tool and moves to comply.  Because the robot is doing the moving, there is no hand tremor, and the robot’s motion may be otherwise constrained by virtual fixtures to enforce safety barriers or otherwise provide guidance for the robot.  However, any robot mechanism has some small amount of compliance, which can affect accuracy depending on how much force is exerted by the human on the tool.  In this project, the student will use existing instrumentation in our lab to measure the displacement of a robot-held tool as various forces are exerted on the tool and develop mathematical models for the compliance.  The student will then use these models to compensate for the compliance in order to assist the human place the tool accurately on predefine targets.  We anticipate that the results will lead to joint publications involving the REU student as a co-author.

Required Skill: The student should be familiar with basic laboratory skills, have a solid mathematical background, and should be familiar with computer programming.  Familiarity with C++ would be a definite plus, but much of the programming work can likely be done in MATLAB or Python.

Faculty Advisor: Prof. Russell Taylor

 

Project 22: Voice Interfaces for Surgical Robot Systems
Description: The goal of this project would be development of suitable voice interfaces for surgical robots such as the da Vinci surgical system. Surgical robots operate in an information rich environment and often have several different operating behaviors.  Although the actual motion of the robot is best controlled by conventional teleoperation or hands-on cooperative control, selection of the appropriate behavior, control of information displays shown to the surgeon, or other information-based interactions may be require the surgeon to communicate information without affecting what his or her hands are doing.  Voice is a natural communication modality for this sort of interaction. In this project, the student will adapt a commercial voice recognition system to provide simple voice interactions with a surgical robot and will demonstrate this capability with one of our surgical robot systems.

Required Skill: The student should be a very strong C++ programmer.  Familiarity with voice recognition or robotics would be a plus.

Faculty Advisor: Prof. Russell Taylor

 

Project 23: Reconstruction of Light Position from Shape of Illumination
Description: In Vitreoretinal surgery the surgeon uses a handheld instrument equipped with a fiber-optic light pipe to illuminate the retina from inside the eye. On microscopic images only the tip of the light pipe may be visible but not the shaft, thus the location of the light pipe cannot be reliably determined by just looking at the instrument itself. In this project we aim to determine the light pipe’s position and orientation by looking at the shape of the illumination pattern on the surface of the retina.

Required Skill: Image processing, Programming (Matlab or C++)

Faculty Advisor: Prof. Russell Taylor, Balazs Vagvolgyi

 

Project 24: Language of Surgery Description: Work with surgeons, residents, and engineers in this project where we model how surgeons learn and perform surgical procedures using the da Vinci Surgical System. This project may involve labeling of activities in surgical videos, developing algorithms for analyzing surgical skill, and evaluating methods on several datasets we have collected of various training tasks. For more information see our website: http://cirl.lcsr.jhu.edu/ Required Skills: Python or Matlab. (Preferred: Machine Learning and/or Computer Vision experience)

Faculty Mentor and Other Mentors: Gregory Hager, Anand Malpani (anandmalpani@jhu.edu), Colin Lea (colincsl@gmail.com)

 

Project 25: Simulation Development for da Vinci (Robotic) Surgery Training Description: This project is motivated by the need for an open-source based simulation framework for training in robotic surgery. An open-source framework allows easy access to information on the environment in the training task allowing machine learning algorithms to perform automated assessment of surgical skills more reliably. As the environment including the tools and other interacting objects in the task are being simulated by the computer, automated tools can get 100% accurate (ground truth) information on positions and orientations of these objects. Required Skills: * Object Models development: SolidWorks or similar (intermediate) * Packing the simulation task: C++ (intermediate) AND Python (basic) AND Robotics – Kinematics (beginner)

Faculty Advisory and Other Mentors: Gregory Hager, Anand Malpani (anandmalpani@jhu.edu)

 

Project 26 :Multi-modal Activity Recognition:
We have recently developed algorithms for modeling how humans interact with the environment both in industrial and surgical settings. In this project you will setup infrastructure for evaluating these algorithms on recent benchmarks in the computer vision community. This project has the following goals: (1) create Python tools for experimenting with datasets (e.g. Cornel CAD 120, Georgia Tech Toy Planes) (2) develop a set of appropriate features that model human-object interactions (3) evaluate our algorithms on these datasets and compare with those in the literatu

Faculty Mentor and Other Mentors: Gregory Hager, Colin Lea

 

Project 27: RGBD-based Object Recognition:

We are working on a robotic system for task automation and activity recognition in collaborative human-robot tasks. We would like to expand the vision pipeline by incorporating object recognition capabilities from Point Cloud Library (PCL). This project has the following goals: (1) create wrappers for existing object recognition modules in PCL (may require C++ experience) (2) expand our pipeline for simultaneously recognizing and tracking objects (3) evaluate these approaches for real-time use in our system
Faculty Mentor and Other Mentors: Gregory Hager, Colin Lea
Suggested requirements: * Python experience (NumPy+OpenCV preferred) * Computer vision or machine learning experience

 

Project 28: Object Recognition
The project aims to develop object representations (models that capture prior knowledge about how the object looks like under varying viewing conditions) and techniques to perform the tasks of object detection, categorization, image segmentation and pose estimation in a fast and efficient manner. We are developing a graph-theoretic approach in which different levels of abstractions, such as pixels, superpixels, object parts, object categories, their 3d pose, relative configuration in the scene, etc., are represented as nodes in a graph. Contextual relationships among different nodes are captured by an energy defined on this graph. In this energy, bottom-up costs capture local appearance and motion information among neighboring nodes. Each of the tasks corresponds to terms in the energy function (the top-down costs), which is then solved in a joint manner. We are also developing algorithms based on branch and bound (pose estimation task) and graph cuts (image segmentation task) for minimizing the energy, and methods based on structured output learning (structural SVMs) to learn its parameters.

 

Internship Goals. As part of the project, the intern will help enhance our current framework for object recognition by improving the model to capture more sub-categories, develop models for more object categories and design algorithms to utilize these models for various vision tasks. The intern will be exposed to current research in the area of Object Recognition and Scene Understanding. He/she will read a lot of literature on a variety of topics like image representation, clustering, classification and graphical models. The intern will implement algorithms in Matlab/C++ and test them across various datasets. The intern will present their work to other graduate students and professors and will potentially be able to publish their research in computer vision conferences and journals. This project will help the intern gain a good understanding of challenges and research opportunities in the area of Object Recognition and Scene Understanding.

Required Skills: Experience in C++ and MATLAB coding and familiarity with image processing, computer vision, or statistical inference is a plus.

Faculty Advisor: Dr. Rene Vidal

 

Project 29 : Application of Linear Dynamical Systems in Automatic Classification of Human Actions in Videos
Project Goals. The high level goal of this project is to develop algorithms for recognizing human actions in videos or more generally pattern recognition in high-dimensional time-series data. We model the time evolution of video frames or features extracted from frames as the output of a Linear Dynamical System (LDS) stimulated with a certain type of input (e.g., white Gaussian noise or a sparse of train of spikes). This brings about certain statistical and computational advantages. In the recognition step, we perform classification in a specific space of LDSs. In other words, we perform recognition in the space of models rather than on raw data. This approach involves defining novel and easy-to-compute distance between LDSs. The geometry of the space of LDSs is a non-Euclidean geometry which our approach takes into account.
Internship Goals. As part of the project, the intern will work alongside PhD students and post-docs to develop novel algorithms for activity recognition using linear dynamical systems. In the first step, the intern will familiarize herself/himself with our existing algorithms and system. The intern also will test our system on various databases and become familiar with its strengths and weaknesses. In the next step, the intern will come up with ways to improve our system (this could be based on other existing machine learning algorithms or based on new ideas specific to our methodology). The internship provides a unique opportunity to become familiar with cutting-edge methodologies in data sciences and to learn about how advanced mathematics is being used to give solutions to real-world problems. It is expected that the results obtained in this internship will appear as one or more original research papers in reputable conferences or journals. As part of the group, the intern will experience first-hand a rigorous and rewarding research environment. Required qualifications include strong background in calculus, linear algebra, and MATLAB programming. Desirable qualifications include a course on linear dynamical systems or linear control systems, and familiarity with C++ and GPU programming. It is expected that the intern will have a curious and creative attitude and will be eager to learn new things.
Faculty Advisor: Dr. Rene Vidal

 

Project 30: Analysis of Diffusion Magnetic Resonance Images
Project Goals. To make DMRI beneficial in both diagnosis and clinical applications, it is of fundamental importance to develop computational and mathematical algorithms for analyzing this complex DMRI data. In this research area, we aim to develop methods for processing and analyzing HARDI data with an ultimate goal of applying these computational tools for robust disease classification and characterization. Possible project areas include:

1. ODF Estimation: To develop advanced algorithms for computing accurate fields of Orientation Distribution Functions (ODFs) from HARDI data.
2. Fiber Segmentation: To develop advanced algorithms for automatically segmenting HARDI volumes into multiple regions corresponding to different structures in the brain.
3. HARDI Registration: To develop advanced algorithms for the registration of HARDI brain volumes to preserve fiber orientation information.
4. HARDI Feature Extraction: To develop methods for extracting features from high-dimensional HARDI data that can be exploited for ODF clustering, fiber segmentation, HARDI registration and disease classification.
5. Disease Classification: To develop advanced classification techniques using novel HARDI feature representations to robustly classify and characterize neurological disease.

 
Internship Goals. In our lab, the intern will work with a PhD student to complete a project within an area(s) mentioned above. The intern will read a number of research papers on DMRI and the state-of-the-art, and will learn an understanding of the problem, its applications, and the techniques involved to tackle it. There are two aspects of the research that may be of interest to the applicant. One is a more theoretical aspect that involves developing mathematical theories to improve existing frameworks. The second is a more computational aspect that involves more image processing, analysis, and algorithm implementation in MATLAB or C++. An applicant with some interest and experience in both areas is most favorable, but it is possible for an applicant to be interested in working on only one of the aspects as well. At the end of the internship period the student will present their work to other graduate students and professors and will potentially be able to publish their research in medical imaging conferences and journals. As becoming part of the Vision Lab, the intern will experience first-hand a rigorous and rewarding research environment with a highly collaborative and supportive social element.
Required Skills: Experience in MATLAB or C++ and familiarity with image analysis or processing is a plus. Mathematical maturity is also favorable.
Faculty Advisor: Dr. Rene Vidal