had originally focused on providing
a set of easy-to-reuse 3D graphics rendering and interaction operations in order
to enable a 3D interactive graphics developer to write programs that take
advantage of rapidly growing powerful graphics hardware featuers
with minimal efforts.
The development of applications in the 3D
domain requires writing programs from scratch in most of cases.
This makes the process difficult and time-consuming. The results
of this project should, hence, be of
particular interest to a wide range of 3D application developers,
the experts in 3D graphics system to the novices that
can appreciate the interactivity of their own 3D program
without requiring deep knowledge in the rendering and intereaction
To identify a common set of objects and methods
necessary to produce interactive 3D applications, our investigation
has been oriented in the direction of devising a clearer
frontier between the application-dependent information and
the interaction mechnisms.
More than the features provided by the well-known interface to 3D
OpenGL, whose ultimate goal is to generate a photorealistic
representation of a 3D scene on the 2D screen, our goal is to
go beyond the conversion of 3D models into 2D array of pixels.
Through interactions with the drawn pixels on the screen via
a mouse, we would
like to manipulate the objects in their own "3D world" as
we have them in our hand.
Differently from architectures, such as the OpenInventor, that
integrate the application data with
the rendering procedures so that the interactions with relatively
complicated shapes would benefit from the existence of geometric
modeling techniques such as intersection and deformation, we
propose in this
project to reuse the application-independent data generated along
the graphics rendering
pipeline, such as differential geometric properties, for providing
sophisticated 3D interaction facilities.
On the basis of our proposal, we developed a GPU-based
architecture for performing basic interaction functionalities, such
as cursor picking and cursor snapping, without requiring a copy of the
surface model in video memory. This is because that
the surface geometry with which we interact may either not exist in the
system memory, such as 3D images, or be
deformed on the vertex shader, on the fragment shader, or on both
along the rendering pipeline. The results encouraged us to
move to medical applications and to implement on top of
Spvolren a curvilinear reformatting tool and
a measuring tool for 3D medical scans in the DICOM.
Curvilinear reformatting consists in peeling off layers parallel to the brain surface like peeling an onion such that the cortical structure can be gradually visualized from the view of the superior surface of the brain after the removal of the meninge and the surface vasculature. In our implementation a user can selectively paint the region to be peeled on the scalp and control the peeling depth. The structute that bridges the painted pixels and the scanned voxels is a triangula mesh. Through the input of a sequence of points on the visible surface via mouse, the length of a piecewise-linear curve may be calculated. Length measurments are voxel-based.
In cooperation with the neuroscientists of Laboratório de Neuroimagem we have developed
three interactive tools that may improve the diagnosis of focal cortical dysplasia, which is the most common causes of pharmacoresistant epilepsies: an interactive editor for
transfer functions that map the physical signals onto graphical attributes (color and opacity), a volumetric probe with which one can highlight any spatial region of interest with distinguishing colors, and a controllable lens in
terms of its magnification factor and its placement to provide an alternative view of the data in a specific display area. From usability tests, we have realized that slice-based investigation is still the preferred one.
The demand for arbitrary angulation in multiplanar reformatting to get a slice view that cannot be scanned leads us to develop an interactive tool that allows a neuro-scientist to reformat any scanned volume along a flat plane in arbitrary orientation. The tool was so designed that a user has control both in 2D and 3D views.
In parallel, we also research rigid multi-modal image registration and fusion in order to build a multi-modal interactive visualization. This is because that multi-modality imaging provides a way to generate non-invasively and simultaneously structural, anatomical and functional views of a brain. Because of its potential application on non-rigid registration, we implemented a technique that is based on mutual information. Even we adopted the well-recommended solution for each stage, we observe that the technique still fails to align some subtle details.
Once pre-alignment had not been deeply explored by the community, we decided to investigate its effects on the registration accuracy. From theoretical point of view, the closer are the floating and reference volumes, the more precise will be the convergence of a floating volume to the reference one in our optimization algorithm. In practice, we asked ourselves which algorithm could robustly recognize any head's shapes and pre-align them in their native space? We devised an algorithm for which an expert takes an important role. An expert only should provide a pair of correspondences and the machine does the rest of the job.
These results, together with the improvements we did on our curvilinear reformatting algorithm, led us to a new idea: to combine them for visual analysis of the anatomical and functional neuroimages in order to detect subtle dysplastic lesions. For efficient mult-volume rendering we devised a novel approach to correctly fetch the complementary multimodal data from the scanned volumes that are not necessarily axis-aligned, with the same size and with the same resolution. Differently from the previous works our GPU-based implementation does not require any kind of pre-classification.