Asiya

Asiya

Translation quality aspects are heterogeneous and diverse, involving, in general, many different linguistic dimensions. However, most automatic evaluation methods in use today rely on partial quality assumptions, such as lexical similarity. This introduces a bias in the development cycle which in some cases has been reported to carry very negative consequences. In order to tackle this methodological problem, we explore a novel path towards heterogeneous automatic MT evaluation. We have compiled a rich set of specialized similarity metrics operating at different linguistic levels (lexical, syntactic and semantic). We have also studied how the scores conferred by different metrics may be integrated into a single measure of quality, without having to adjust their relative importance.

Asiya has been designed to assist both system and metric developers by offering a rich repository of metrics and meta-metrics. Asiya has been developed at TALP Research Center NLP group , in Universitat Politècnica de Catalunya, as an evolution, extension, refactoring, and finally a replacement for its predecessor, IQMT.

Join our Forum!

Check or post messages in the Asiya Forum: http://nlp.cs.upc.edu/redmine/projects/asiya/boards

News Feed:

Download

Latest development version may be downloaded through subversion.
Everyone is allowed to check out (User: reader, Password: reader).

• svn co http://svn-rdlab.cs.upc.edu/subversion/asiya/public asiya

Asiya is also open to public contribution. If you feel like helping us in the development, please, e-mail us at so we grant you with the necessary permissions.

The Asiya framework is released under the GNU Lesser General Public License (LGPL) of the Free Software Foundation. This means that it may be linked to and used by commercial software packages. But the license also enforces that any changes or improvements made to the library must be redistributed under LGPL terms.

Thus, if you improve the software, either adding new functionalities, fixing bugs, or adding new metrics, you can not distribute them under different conditions than those stated in the license (i.e. freely and with no usage restrictions).

If you want that your changes and improvements become useful to many other people using this free software, please contact us:

Documentation

This report describes the installation and usage of the Asiya Open Toolkit for Automatic Machine Translation (Meta-)Evaluation. Asiya offers system and metric developers a text interface to a rich repository of metrics and meta-metrics. [Technical Manual (version 3.0)]

Asiya Web Service

The Asiya Web Service is intended to facilitate the remote usage of Asiya without the need for downloading and locally installing all the modules. It allows to access the application from any remote client running on any platform or developed using other tools. Thereby, the service eases the integration of \Asiya\ as part of other applications that may be working on heterogeneous platforms.

On the other hand, the Asiya tool is computationally demanding, especially when parsing is needed. This makes impractical the concurrent execution of several requests in a regular server. In order to be able to handle big datasets and multiple Asiya executions, the service architecture includes an engine to submit jobs to the computing cluster.

The service is currently running in the following server:	http://asiya-faust.cs.upc.edu
The user manual gives the details about the operations exposed and the parameters required:
A simple HTML client and sample data showing how to access the service: