New feature: now blast xml parser can read either raw xml or gzipped xml files. Also HomologList will use xml.gz files only as cache files

Small fix to allow for underscore in blast query id

Small fix: moved up the naming of blast cache file

Small fix to be sure that the uniprot ver reported in error message is the actual one that the japi server returned, at the moment they seem to be having issues and 2 calls to the server return different versions!

Fixed important issues with searching and alignment of homologs: 
- ids and coverage were calculated wrongly, based on all BlastHsps for a hit. Now based on single Hsps
- fixed bug in BlastHsp.getQueryCoverage: was miscalculating the coverage by 1 unit
- the useHspsOnly parameter in alignment didn't make much sense. We need always to use the hsp segments only, extending to the full sequence is dangerous and in any case does not make sense at all if the clustering is done in hsps only

Several changes related to blast:
- now supporting blast+ blastp. 
- blast xml parser can now ignore the DTD url, avoiding unnecessary network access if desired
- refactoring: legacy blast related stuff is now refactored to contain "legacy", runBlastp refers to blast+
Also updated uniprot jar.

Now supporting clustalo as well as tcoffee for alignment of homologs

Now logging t_coffee total runtime

Minor fixes in entropy calculation

Now checking that alignment from cache contains actually what we are expecting

Solved a couple of small issues in HomologList

Fixed bug: for a uniref100 cluster member, was taking the representative's uniprot id/tax id instead of the member's. Database was not correctly modelled -> need also the tax  ids for cluster members (and then to query the tax ids from members and not from representative)

New feature: now reducing redundancy via clustering with blastclust

Improved a bit the skimming strategy to be a bit less naive: now it skims more from the very high identities [e.g. it seems to improve case 3b37_1, a xtal contact]

Writing fasta tag with subinterval also for query

Now we can do alignments using hsp regions only (instead of always full sequences)

Fixed bug: redundancy (duplicate) elimination was done on whole uniprot sequences and not on HSPs

Introduced identicals elimination, removed the taxonomy grouping in removeRedundancy (didn't make any sense anymore without coding sequences)

Changed how uniprotver is read

Fixed issues with homologs with a uniparc id as reference, now the reference can be either uniprot or uniparc

Fixed bug in uniref xml parser: was not taking the right uniprot representative for old style (e.g. ver 1.0) xml files. They also contain several uniprot ids sometimes: first one being the active representative and the remaining being the inactive ones.

Implemented retrieval of uniprot KB data from local database

Restructured how the UniprotConnection is used. Now all uniprot connection stuff done within the class

Now the reference entry in HomologList is also a UnirefEntry. Not using anymore UniprotEntry

Major change: Homolog and HomologList are now using the new class UnirefEntry. Removed a lot of the code related to Ka/Ks calculation and EMBL CDS retrieval

More complete parsing: cluster members and inactive ids

Some refactoring

Implemented caching of alignment files

Adding flag to use uniparc or not

Now supporting both uniprot and uniparc entries in HomologList

New methods for filtering by domain of life

Now UniprotHomologList contains a full list and a filtered list, making it more flexible

Now catching OutOfMemory errors and continuing in redundancy removal procedure

New method to produce statistics of the sequence entropies variability

Fixed minor bug: now will re-blast only if maxNumSeqs above 500 and more hits requested

New feature: now can specify a subinterval in UniprotHomologList

Fixed issue with uniparc and uniprot isoform ids

Modified the uniref regex to also capture uniprot isoforms identifiers

Improved the blast cache checking

Now can pass option -v to blast (max number of hits reported)

Fixed logging and error handling of tcoffee: now it will really log errors to log file, temp files won't be deleted when exit!=0, command line is logged

Fixed minor issue: wasn't logging an error through logger

More logging

Now blasting against uniref also supported in UniprotHomologList

Now explicitely specifying number of threads in TcoffeeRunner

New method for getting URL of uniprot entry

Some new methods for disulfide detection and other minor things

MAJOR CHANGE: now able to read and treat properly HETATMS and nucleotides from PDB entries. All tests pass. Anyway surely there will be some bugs to iron out still.

MAJOR INTERFACE CHANGE: the main PDB data loading interface has changed. Now all loading of data occurs through PdbAsymUnit. Refactoring of some classes: Pdb is now PdbChain.

Now detecting type of sequence: protein/nucleotide

Fixed bugs in method for reading from PIR file; added script to convert from PIR to Fasta

Now throwing InterruptedException. Refactor BlastErro to BlastException

Refactoring xxxError to xxxException

Important change: now the PDB file reader will try to read and fix the numbering of PDB files. Whenever the alignment is wrong it will realign and renumber using the jaligner package. The result of reading original PDB files now will be the same as that of reading CIF files, including proper mapping of classical PDB numbers to SEQRES residue serials (as in CIF). 
There is one change of behaviour in comparison to before: when no SEQRES present the sequence is taken to be that of ATOM lines instead of padding it with Xs.
Still in this version the re-alignment is not perfect as there are some times when ambiguities occur and they are not solved (e.g. in 2nwr where alignment in an unobserved loop can be at two possible places for a GLY). That is anyway a rather minor problem (coordinates are still fine, just the chain is not ordered correctly at 1 or 2 points) and rare (~1% of files)

Renamed xxxxError classes to xxxxException as they should be.

Catching if the server returns non-requested records and throwin IOException for it.

Fixed bug: was not checking whether the uniprot japi was actually returning all requested records for ids given when using getMultipleEntries. Now checking, logging it and removing the not-found ids from the homolog list.

Added Serializable interface to many classes. To be able to serialize classes containing objects of these types.

Renaming all xxxxError named exceptions to xxxxException. Long overdue thing (was historical because it came from python). Learnt just recently that java does have a concept of Error too.

Fixed important bug: we didn't model the blast data properly. A hit is composed of several hsps. Now our classes following that data model properly (before we treated a hit as an hsp).

Better docs

Now storing the query coverage cutoff.

Now throwing MatchNotFound exception instead of catching it.

Overrides removed

Fixed buf: EMBL DB fetch returns sequences now in upper case, we were not converting properly to lower case in one place, now should be fixed.

Now possible to not retrieve any CDS data

Using the new UniprotVerMismatchException instead of IOException

New exception for mismatching uniprot versions

Some minor updates

Now using apache commons logging

Now catching the case when the best translation contains stop codons. This is usually due to a wrong genetic code, but for the moment it's still very difficult to know the genetic code from encoding organelle+organism taxonomy, so this is the temporary solution.

Fixed bug: not allowing anymore the presence of gaps when choosing the representative CDS to uniprot match. Now the nucleotide alignments should be always correct (before they were shifted if there were gaps in CDS-to-uniprot)

Now returning null for the representative CDS when the gene encoding organelle is not nucleus/plasmid. Before the whole program would stop, which wasn't ideal. Eventually we will need to use the proper genetic code when needed.

Allowing plasmids as gene encoding organelles without having to treat them especially as they don't have a different genetic code.

Reverting change made in last revision. It intended to fix a bug for a particular test case (don't know anymore which one!) but it actually broke most other cases.

Fixed bug, wasn't computing the nucleotide alignment correctly (was missing the last codon)

Implemented a simple skimming strategy. Put back some debug level logging to info level, logging aint' easy!

And all other amiguous codons

Another ambiguous nucleotide letter

Fixed bug: was null pointing with null cache file (which is a valid value)

Changed level of some logging messages

New removeRedundancy method

Implemented logging with log4j library

Now can write sequences with or without query

Now can filter also on query coverage

Changed the order of checks in isReferenceSeqPositionReliable

New methods to check for reliability of positions with respect of the CDS matching

Now nucleotide alignment contains reference sequence as well. Moved entropy and kaks calc methods to UniprotHomologList.

Fixed bug: now nucleotide alignment is in same order as protein alignment

Fixed bug: when retrieving (or reading from cache) embl cds sequences and embl dbfetch doesn't have a certain identifier, then we were adding nulls to the list of emblcds sequences of the UniprotHomologList (resulting in a null pointer down the line)

Now checking the sequence identity of the representative CDS to be above a minimum value.

New ambiguous nucleotide w.

Fixed bug: getNucleotideAlignment was nullpointing when encountering a null return from getRepresentativeCDS() (which happens whenever there is no representative CDS for the particular UniprotEntry). Now checking for nulls before using the homolog in the alignment.

New SelectonRunner class. More taxonomy information retrieved from uniprot.

New code to obtain the nucleotide alignment by mapping the aminoacid alignment to the nucleotide sequences.

Improved the selection of representative CDS

Some refactoring

Now implementing HasFeatures

New class UniprotEntry. Things make a lot more sense now...

Now checking also for ambiguous nucleotide codes (n, m) when translating. Changed implementation of translation by using the new Codon class.

Now checking codon length before translating

Now properly translating all frames and finding the best translation match in checkEmblCDSMatching. New class doing all the matching of protein to CDS.

New feature: code for dna to protein translation

Now getting also gene encoding organelle from uniprot api

Implemented caching of blast in UniprotHomologList

Now reading uniprot version from reldate.txt and checking against the uniprot api version

Implemented caching for EMBWSDBfetchConnection

Fixed bug: was nullpointing at retrieveUniprotKBData because of a single uniprot id can have multiple blast hits and was using the uni ids as unique identifiers => the lookup map was failing. Now the lookup map is from uniIds to lists of homologs(hits)

Now explicitely specifying output dnd file in tcoffee so it can be removed by the calling program.

Fixed bug: was nullpointing when the letter encountered in counting the column was unknown.

Implemented reduced alphabets in AminoAcid class and modified entropy calculation to allow reduced alphabets.

Some new methods

Fixed bug with '-' embl cds identifiers.

Implemented profile counting and entropy calculation for alignment.

Also retrieving uniprot sequence in UniprotHomolog.retrieveUniprotKBData()

Implemented methods to run a tcoffee alignment. Now also retrieving uniprot sequences via the uniprot api.

New methods to retrieve uniprot data and embl cds for a single homolog

Removed code left over from debugging.

Minor changes, couple of new methods.

Minor fix to java docs

removing @Override annotations which were causing compilation errors for me

New classes UniprotHomolog and UniprotHomologList to contain a set of homologs of a given sequence, including the blast hit data, embl cds coding sequences etc.

Uniprot and embl ws dbfetch connections can now do multiple entries per request. Update to parse also the hit_def tag in blast parser.

Fixed some docs and indentation

refactoring: removed a warning in MultipleSequenceAlignment and deleted obsolete directory src/tests

refactoring: moved many many classes to more appropriate packages; created new packages owl.core.sequence.alignment, owl.core.structure.alignment, owl.core.structure.features

refactoring: renaming proteinstructure to structure and tools to util; moving connections,features,runners,sequence,structure,util to owl.core

Reorganised the project with a src folder for java source files.
Added a jars dir with all jars needed for the project.
Added .project and .classpath pointing to relative path of jars. 
The project should now work out of the box after a check-out with eclipse. No need to setup external jars or anything.

some cosmetic changes

New methods for writing sequence in Fasta format

New method contains().

Fixed bug: was not doing correctly the sanity checks in getAlignmentFullSeqs

white TO talyn

white TO talyn

New method in BlastHit to transform the BlastHit local alignment into one that contains the full sequences of subject and query.

Rewrote the secondary structure handling in Alignment. Also changed first parameter in writeFasta to be a PrintStream not an OutputStream

New static method in BlastHit to get subjectId from templateId
New member source in TemplateList

New method getAlignmentWithTemplateIDtag

BlastHitList: new method getHit(subjectId), a new lookup HastMap member added for that.
TemplateList: new method get(i)
Sequence: new convenience method writeToFastaFile

Now cleaning up correctly paul temp files
Now closing tcoffee's log properly

Now we can run blast with multi threads option

Initial commit, tcoffee runner for sequence to profile alignment

Now BlastHit printing works correctly for query identifier strings of any length

Using the new XML parser in TemplateSelection.
New method getQueryCoverage in BlastHit

New tests package (should be a folder and under it packages, but for now we can't do that until we move all source code to a src folder).
Moved all existing tests to appropiate tests packages.
New test PdbParsersTest: does cif against pdbase at the moment.
Adde equals method to SecStrucElement
Added some getters to Pdb

New BlastXMLParser together with BlastParsersTest Junit test for both xml and tabular parsers.
Added some new fields in BlastHist and BlastHitList.
Blast running methods in BlastRunner now take a new parameter: noFiltering.

Created getters for missing fields in BlastHit. 
New method removeAll in TemplateList.

added static method to print a ruler with sequence numbers

Fixed bug: now Sequence class considers '.' to be part of the fasta headers (in agreement with what we do in Alignment class). The 2 FASTAHEADER_REGEXes should be always kept in sync.

New class PhiPsiAverager to get consensus of phi/psi angles from a TemplateList and an Alignment. Tested with a few examples and seems to work. The wrapping of angles at 180/-180 is not yet taking into account, i.e. if an interval falls in the region just below 180 and just above -180, no consensus will be found.
Pdb: added some checks to methods getPhi/Psi so that it doesn't fail when there's no coordinates.
Changed yet again the design of TemplateList/Template. Now loading of PDB data happens upon call of the loadPDBdata method. Changed dependencies accordingly.

Fixed bug: writeClusterGraph shouldn't try to do anything if there's only one template.

added automatic conversion of matrix file to ps visualization (using script plot_simmatrix.sh)

Now skipping lines with <PRE> tags.

Changed default gravity value and default IDENTITY_SCORE for similarity graph

Fixed bug in BlastUtils.writeClusterGraph: was not working at all. Wrong curly brackets, was still using a GDT cutoff instead of rmsd and was never getting Pdb correctly out of the Template.
Two new members in Template: GTGHit and BlastHit, so we can reference back to the hit if needed (like if we want to get the evalue/score). Got rid on MySQLConnection as a member, not very nice design, now a parameter for the specific methods that need to query for PDB data.

put test code for writeClusterGraph in main method of BlastUtils

Now writeClusterGraph uses rmsd for similarity measure instead of GDT (maxcluster was failing almost always with GDT)

Yet another format check

Added one more file format check.

Fixed bug: wasn't catching wrong format in GTG output

Fixed bug: now catching when the mapping of observed residue sequence serials to internal residue serials is out of range, which happens in some extremely rare cases (because GTG uses pre-remediation data)

Added code to be able to filter out template ids (pdb codes) by maximum release date.
Some improvements in Hit classes.

Added methods to be able to print "graphical" output for GTGHits, all equivalent to the methods in BlastHit
Added compare() and writeIdsToFile() in TemplateList.

Initial commit of GTG parsing classes.
Added method in PdbasePdb to map observed residue sequence serials to internal (cif) residue serials

Now checking wheter read sequence contains spaces and throwing FastaFileFormatError

Moved method to writeSeqs to Sequence class.
Tidied up the help text in averageGraph

Fixed the fasta header regex to take only first token of header as sequence name (same behaviour as in Alignment class)

merged in some changes (now writing similarity matrix in blast utils)

Added code to run psipred.
Fixed a few minor bugs.

New classes Template, TemplateList to be used in homology modelling pipeline.
Modified BlastUtils to use TemplateList class

added BlastUtils.writeClusterGraph(String[] templateIds, outFile)

New methods to get the template ids and write them to file

Made printing hits with graphical overview the default using print() method. Print() takes no parameters, but BlastHitList.setQueryLength() has to be called once to set the length of the query. Introduced new printSome() function to output only the first n hits.

added class Sequence, added ascii-art output of blast hits to BlastHitList

Added a main method for testing

New package sequence. Initial commit of classes to run blast and parse its output.