[Pipet Devel] Open DX as a loci skeleton?

[Harry Mangalam]

Brad Chapman wrote:
> 
> > Not to make everyone heartily sick of this thread, but it looks like
> the DX
> > arch CAN be made CORBA-aware with relatively little (hah!) effort.
> If so,
> > that would make it a
> > viable candidate for MY immediate use, and it sounds like this might
> be of
> > wider interest as well.
> >
> > I need to look at the code mentioned with one of our CORBA
> programmers to
> > see how easy it is to in fact do this.
> 
> Harry,
>     This is really great news, and I for one would be really
> interested to hear about how things go with your attempts to do this.

I'll certainly keep the group informed as to what I'm doing.  The project
on which I'm working has precious little slack time built in, but I'll be
using this as an opp to start to learn the actually coding involved in
CORBA.  I suspect that the admin will have little tolerance for another new
approach to the infrastructure, unless I can show them an example of how it
works off the bat.


> Although I don't think all of the applications are of immediate
> interest to people on the list (anyone on the list interested in oil
> and gas mining? :) I think that a lot of the tools provided in open dx
> would be really useful for data mining in general in addition to the
> specific programs for imaging etc.

Well, I dunno - all the things you and I mentioned deal with very large
numbers.  Look at the popularity of BLAST parsers - why do people write
such things?  Because the output is largely meaningless unless it can be
munged into something that poeple can conceptualize better.  That's all
that the Viz capabilities from DX is trying to do.  I'm helping to write
gene expression database analysis tools that will help people see what
their data means in a larger context.  Even in tiny genomes like bacteria,
you have 4.6MB of DNA, ~4300 ORFs, very complex interrelationships on a
gene regulation level, more on a biochemical pathway level.  How do you
hope to have people conceptualize those relations?  I'm hopingthat by using
the kinds of Viz that DX provides, I can at least make a stab at it.  It
WILL be slow and ugly at 1st, but it might work - Incyte has licensed SGI's
Mineset to do this (and sells access at $2M/yr)- I'm hoping to do it ...
considerably cheaper.  One of the things that I'm counting on is that the
underlying Viz technology is being handed to me - I don't have to implement
the actual GL code for flinging things around in 3D, or texture-mapping, or
doing transparency,e tc.  I just have to format the data correctly and
serve it up to the DX Viz engine as an HDF file (or whatever) via CORBA
(eventually); right now all I have to do is format it as an HDF file (which
in large part 'R' can apparently do for me (we're using the R pkg to do
most of the stats).


>     Have you given any thought to trying to integrate your own work
> with open dx with Loci? I have been playing around with Fnorb (a
> python ORB) a little on my own and recently for the Biopython project,
> and if you are interested, I could try and help with getting code for
> Loci to support some kind of idl for talking with Loci. 

I can see where this would be very useful.  If Loci develops as an
analytical pipeline separate from DX or other such approaches, then this
would allow you to graft on the bits of DX functionality that you need. 
Actually, wouldn't this be more or less required to allow Loci to extract
resources from other CORBA-based DB's?  Most of the other large DBs are
going this way NCGR's GSDB, EBI's SRS also has a CORBA interface now, I
think..) and it would be the most powerful, if not exactly lightweight way
of going about it.

>                                                            IMHO, this
> would be of great help to Loci in general (to give it some
> additional functionality) and would also be a great chance for you to
> help out with Loci development (a highly worthy cause!). I'd be
> really interested to hear your thoughts on this, specifically what
> kind of things you would need Loci to do to meet with the needs for
> your project. Just my 2 cents on the topic...

Sure - that's why I posted here.  The whole project is Loci-amenable or
even (for me) a Loci replacement.  I'm not trying to denigrate the Loci
project - it's what got me thinking about this approach, and I've picked up
some great ideas and pointers from the traffic here (in fact, this list is
one of my favorite for user-filtered pointers - if someone posts an
'interesting URL', I'll usually try to track it down; it's almost
guaranteed to be useful) , but we (NCGR) need something that works right
now and even tho DX was born years ago (like unix), it's been
well-thought-out and debugged (like unix) and may, with only a little
meddling at an external interface, support a tremendous amount of the
functionality that we need.  And I personally would rather design nifty
analytical tools than infrastructure. 

Don't forget to read that URL that Gary posted:
http://www.cs.wustl.edu/~schmidt/CACM-lessons.html

Best
Harry




> 
> Brad
> 
> _______________________________________________
> pipet-devel maillist  -  pipet-devel at bioinformatics.org
> http://bioinformatics.org/mailman/listinfo/pipet-devel

-- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 -- mangalam at home.com