- fsa - lists.rwth-aachen.de

integrated networks for ASR
by Owen TN 27 May '05

27 May '05

Dear ALL, 1. I am now trying to build an integrated network (from the lexcion WFST to the languge model WFST). Does anyone have done this work before ? If then, could you please give me instructions as much details as possible ? 2. I read somewhere that we have to take the closure of the union of individual pronounciation transducers. That means we have to keep so many and very small in size of WFSTs, each for the word and its pronounciation . Thank you for you help. Owen __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

compiling problems with fedora core 3 / gcc 3.4.2
by Benoit Favre 11 May '05

11 May '05

in english this time (and no attachment ;) ) System: Linux/Fedora Core 3 FSA version : 0.9.1 Compiler version : 3.4.2 As fsa does not compile as is (1st error log), i tryed to modify the involved lines as suggested by the compiler. This leads in crashing the compiler (2nd error log). By the way, since then, FSA worked fine on an other computer with Ubuntu/GCC 3.3. So, I am planning to replace FC3 with Ubuntu. I also explored the python bindings (the thing i am really interested in) but it does not seem to be possible to manipulate arcs and states directly. Is this functionnaliry planned in next versions ? Benoit. $ g++ -v Reading specs from /usr/lib/gcc/i386-redhat-linux/3.4.2/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-java-awt=gtk --host=i386-redhat-linux Thread model: posix gcc version 3.4.2 20041017 (Red Hat 3.4.2-6.fc3) $ make Rules.make:99: warning: overriding commands for target `install' Makefile:43: warning: ignoring old commands for target `install' ******************************************************************************** Project: The RWTH FSA Toolkit ******************************************************************************** --- snip --- make -C src build make[1]: Entering directory `/local/favre/devel/python/fsa-0.9.1/src' ../Rules.make:99: warning: overriding commands for target `install' Makefile:30: warning: ignoring old commands for target `install' make -C Core libSprintCore.linux-intel-standard.a make[2]: Entering directory `/local/favre/devel/python/fsa-0.9.1/src/Core' compiling Application.cc In file included from ../../src/Core/TextStream.hh:24, from XmlStream.hh:26, from Configuration.hh:21, from Configurable.hh:20, from Component.hh:21, from Application.hh:22, from Application.cc:21: ../../src/Core/Types.hh:79: error: an explicit specialization must be preceded by 'template <>' In file included from Channel.hh:21, from Component.hh:22, from Application.hh:22, from Application.cc:21: ../../src/Core/Hash.hh:19:30: ext/stl_hash_fun.h: No such file or directory Application.cc:220:2: warning: #warning is a GCC extension Application.cc:220:2: warning: #warning "This is a workaround for possible bug in LIBC 2.3.2. We are waiting for an answer on our bug report." make[2]: *** [.build/linux-intel-standard/Application.o] Error 1 make[2]: Leaving directory `/local/favre/devel/python/fsa-0.9.1/src/Core' make[1]: *** [Core] Error 2 make[1]: Leaving directory `/local/favre/devel/python/fsa-0.9.1/src' make: *** [build] Error 2 $ vi src/Core/Types.hh < src/Core/Types.hh:79 class NameHelper<std::string> : public std::string { > src/Core/Types.hh:79 template <> class NameHelper<std::string> : public std::string { $ vi src/Core/Hash.hh < src/Core/Hash.hh:19 #include <ext/stl_hash_fun.h> > src/Core/Hash.hh:19 #include <ext/hash_fun.h> $ make Rules.make:99: warning: overriding commands for target `install' Makefile:43: warning: ignoring old commands for target `install' ******************************************************************************** Project: The RWTH FSA Toolkit ******************************************************************************** --- snip --- make -C src build make[1]: Entering directory `/local/favre/devel/python/fsa-0.9.1/src' ../Rules.make:99: warning: overriding commands for target `install' Makefile:30: warning: ignoring old commands for target `install' make -C Core libSprintCore.linux-intel-standard.a make[2]: Entering directory `/local/favre/devel/python/fsa-0.9.1/src/Core' compiling BinaryStream.cc BinaryStream.cc:20:2: warning: #warning is a GCC extension BinaryStream.cc:20:2: warning: #warning "CodeSmell: wrapper and file stream roles should be separated: BinaryWriter/Reader vs. BinaryStream" BinaryStream.cc:105: warning: specialization of `template<class T> bool Core::BinaryOutputStream::write(const T*, std::streamsize)' in different namespace BinaryStream.hh:136: warning: from definition of `template<class T> bool Core::BinaryOutputStream::write(const T*, std::streamsize)' BinaryStream.cc: In member function `bool Core::BinaryOutputStream::write(const T*, std::streamsize) [with T = bool]': BinaryStream.cc:106: internal compiler error: in comptypes, at cp/typeck.c:932 Please submit a full bug report, with preprocessed source if appropriate. See <URL:http://bugzilla.redhat.com/bugzilla> for instructions. Preprocessed source stored into /tmp/ccArjCeU.out file, please attach this to your bugreport. make[2]: *** [.build/linux-intel-standard/BinaryStream.o] Error 1 make[2]: Leaving directory `/local/favre/devel/python/fsa-0.9.1/src/Core' make[1]: *** [Core] Error 2 make[1]: Leaving directory `/local/favre/devel/python/fsa-0.9.1/src' make: *** [build] Error 2

1 0

concat order
by Roy Tromble 11 May '05

11 May '05

I noticed yesterday that the Fsa command-line tool's concat operator works backward from what I expected. It puts the transducer at the top of the stack (what I think of as right-most) at the front of the concatenated transducer. Is this the intended behavior? It seems especially strange since for the operations that take two transducers as arguments, second_ is defined as the top of the stack, and first_ as the position below it. Thanks, Roy

3 2

compiling on fedora core 3 / gcc 3.4.2
by Benoit Favre 11 May '05

11 May '05

i just downloaded fsa and it seems really promising. but, i get a buch of errors while compiling with fc3/gcc3.4.2 : (sorry for the localized messages) -> gcc -v gcc 3.4.2 20041017 (Red Hat 3.4.2-6.fc3) ../../src/Core/Hash.hh:19:30: ext/stl_hash_fun.h : Aucun fichier ou répertoire de ce type -> file does not exist (i replaced it with ext/hash_fun.h which looks good) ../../src/Core/Types.hh:79: erreur: spécialisation explicite doit être précédé par « template <> » -> should add "template <>" at line 79 (so, i added it) BinaryStream.cc:105: attention : specialization of `template<class T> bool Core::BinaryOutputStream::write(const T*, std::streamsize)' in different namespace BinaryStream.hh:136: attention : Ã partir de la définition de «template<class T> bool Core::BinaryOutputStream::write(const T*, std::streamsize) » BinaryStream.cc: In member function `bool Core::BinaryOutputStream::write(const T*, std::streamsize) [with T = bool]': BinaryStream.cc:106: erreur interne du compilateur: dans comptypes, cp/typeck.c:932 Please submit a full bug report, with preprocessed source if appropriate. See <URL:http://bugzilla.redhat.com/bugzilla> for instructions. -> compiler internal error (there i cannot do much...) i attached the preprocessed file output by the compiler (cceZFVUq.out). could you help me a little ;) -- Benoit Favre

3 3

How does minimization work?
by Emilian Stoimenov 09 May '05

09 May '05

Hi Stephan, Should I make anything special in order to minimize my transducer? Mohri states in a paper that minimization of a WFST is a simply minimization of a FSA with the labels on the arcs of the WFST appropriately encoded. Is this encoding done internally in the fsa tool or should I do it separately as in the case with the AT&T fsm library? Or do you do it the other way: invert-determinize-invert? Best, Emilian

1 0

Compiling on Darwin
by araleius 03 May '05

03 May '05

Hello, I haven't been able to compile fsa-0.9.1 on macosx. I don't really get the warnings (mentioned in the README) but some errors concerning malloc.h and thread.h. I've already had problems compiling other packages with malloc.h, but since apple "has included this functionality elsewhere", removing the #include <malloc.h> statements is usually enough. Unfortunately, this doesn't work for thread.h and ost namespace -- for instance -- is not recognized, etc. Has anyone had any experience compiling the toolkit on macosx? Thanks here's the make result: [babayaga: fsa-0.9.1]$ make Rules.make:99: warning: overriding commands for target `install' Makefile:43: warning: ignoring old commands for target `install' ************************************************************************ ******** Project:\t\tThe RWTH FSA Toolkit ************************************************************************ ******** Build host:\t\t myhostname Build date:\t\t Tue May 3 13:04:12 CEST 2005 Build directory:\t /Users/araleius/Software/fsa-0.9.1 Processor:\t\tppc Operating system:\tdarwin Version:\t\t - standard Profiling:\t\tnone Compiler:\t\tgcc / g++ Preprocessor flags:\t-DSTANDARD -DBUILD="" -DPROC_ppc -DOS_darwin -DARCH_darwin_ppc -D__BASE_DIR__="/Users/araleius/Software/fsa-0.9.1" -D_GLIBCPP_USE_C99 -DTIMESTAMP -I. -I./src -I/usr/include -I/sw/include/libxml2 -I/sw/include -I/usr/local/include/cc++2 C compiler flags:\t -funsigned-char -fno-exceptions -Wall -arch ppc -O2 -mcpu=7450 C++ compiler flags:\t -funsigned-char -fno-exceptions -Wall -arch ppc -O2 -mcpu=7450 -fpermissive -Woverloaded-virtual Linker:\t\t\tg++ Linker flags:\t\t-L/sw/lib -lxml2 -lz -lpthread -L/sw/lib -liconv -lm -L/usr/local/lib /usr/local/lib/libccgnu2.a -lz -framework vecLib -lm -ldl -lpthread -prebind -undefined error Installation directories: Base:\t\t\t Binaries:\t\t Includes:\t\t Libraries:\t\t ************************************************************************ ******** make -C src build ../Rules.make:99: warning: overriding commands for target `install' Makefile:30: warning: ignoring old commands for target `install' make -C Core libSprintCore.darwin-ppc-standard.a compiling Application.cc In file included from Application.cc:14: /usr/include/gcc/darwin/3.3/c++/cstdlib:139: error: `_Exit' not declared /usr/include/gcc/darwin/3.3/c++/cstdlib:165: error: `_Exit' not declared Application.cc:20:20: malloc.h: No such file or directory In file included from Channel.hh:23, from Component.hh:22, from Application.hh:22, from Application.cc:21: ../../src/Core/Thread.hh:18:25: cc++/thread.h: No such file or directory In file included from Channel.hh:23, from Component.hh:22, from Application.hh:22, from Application.cc:21: ../../src/Core/Thread.hh:23: error: `ost' is not a class or namespace ../../src/Core/Thread.hh:24: error: `Mutex' is not a class or namespace ../../src/Core/Thread.hh:24: error: invalid base-class specification ../../src/Core/Thread.hh: In constructor `Core::Mutex::Mutex()': ../../src/Core/Thread.hh:26: error: `ost' is not a class or namespace ../../src/Core/Thread.hh:26: error: `Mutex' is not a class or namespace ../../src/Core/Thread.hh:26: error: class `Core::Mutex' does not have any field named `ost' ../../src/Core/Thread.hh: In member function `void Core::Mutex::lock()': ../../src/Core/Thread.hh:27: error: `enterMutex' undeclared (first use this function) ../../src/Core/Thread.hh:27: error: (Each undeclared identifier is reported only once for each function it appears in.) ../../src/Core/Thread.hh: In member function `void Core::Mutex::release()': ../../src/Core/Thread.hh:28: error: `leaveMutex' undeclared (first use this function) ../../src/Core/Thread.hh: At global scope: ../../src/Core/Thread.hh:31: error: `ost' is not a class or namespace ../../src/Core/Thread.hh:31: error: `MutexLock' is not a class or namespace ../../src/Core/Thread.hh:31: error: invalid base-class specification ../../src/Core/Thread.hh:33: error: parse error before `::' token ../../src/Core/Thread.hh:33: error: missing ';' before right brace ../../src/Core/Thread.hh:36: error: `ost' is not a class or namespace ../../src/Core/Thread.hh:36: error: `Conditional' is not a class or namespace ../../src/Core/Thread.hh:36: error: invalid base-class specification ../../src/Core/Thread.hh:48: error: `ost' is not a class or namespace ../../src/Core/Thread.hh:48: error: `Thread' is not a class or namespace ../../src/Core/Thread.hh:48: error: invalid base-class specification Application.cc:220:2: warning: #warning "This is a workaround for possible bug in LIBC 2.3.2. We are waiting for an answer on our bug report." Application.cc: In static member function `static int Core::Application::main(int, char**)': Application.cc:221: error: `M_TOP_PAD' undeclared (first use this function) Application.cc:221: error: `mallopt' undeclared (first use this function) make[2]: *** [.build/darwin-ppc-standard/Application.o] Error 1 make[1]: *** [Core] Error 2 make: *** [build] Error 2 [babayaga: fsa-0.9.1]$

1 0

suppressing xml in output streams
by Roy Tromble 13 Apr '05

13 Apr '05

Hi, I'm running some applications using Fsa by extending Core::Application, and I'm wondering if there's a simple way to suppress the xml and indentation that get printed when I use standard out. It looks like this: <?xml version="1.0" encoding="ISO-8859-1"?> <sprint> </sprint> and everything in between is indented two spaces. Thanks, Roy Tromble

2 1

Composing and determinizing efficiently
by Teemu Hirsimäki 17 Feb '05

17 Feb '05

Hi, again. After getting familiar with the FSA toolkit, it seems to be a very nice toolkit. However, I have one question. When big transducers are composed and determinized, is normally necessary to some additional operations to improve the performance of the FSA toolkit? Or should "compose", "trim" and "determinize" be enough? I have an acyclic pronunciation lexicon transducer (50 input labels, 24362 output labels, 20784 states, 45145 arcs) with disambiguation symbols, and a language model acceptor (24363 labels, 729323 states, 5640013 arcs). Composing them with the AT&T toolkit (using "dmake -a lex -b lm") takes 4m35s with peak memory consumption around 1,4G. The RWTH FSA tool, on the other hand, got up to 11G in 15 minutes after which I stopped the process before it started to use swap. I used the following command line: fsa --progress=yes bin:lex closure bin:lm compose trim determinize \ write bin:composition The system is SuSE Linux 9.0 (x86-64). Any ideas if I am missing something? Perhaps I have to look more carefully that the conversion between formats is ok. At least the number of states, arcs, input-epsilon-arcs and output-epsilon-arcs seems to match. -- Teemu Hirsimäki

1 0

Compiling on SuSE 9.1
by Teemu Hirsimäki 16 Feb '05

16 Feb '05

I managed to compile the RWTH FSA Toolkit on a SuSE 9.1 Linux system, but I did have to make a few changed in the code. I had to add the line #include <cc++/tokenizer.h> in files src/Core/Configuration.cc src/Core/Directory.cc src/Fsa/Storage.cc Then I also had to edit the file "config/os-linux.make" and change the line LDFLAGS += $(shell ccgnu2-config --libs) to LDFLAGS += $(shell ccgnu2-config --stdlibs) as the "--libs" flag did not seem to add enough libraries for the Common C++. In particular, the "-lccext2" flag was missing. The version of the Common C++ library was 1.3.1. -- Teemu Hirsimäki +358 50 3667288

2 1

Corrections to README
by Teemu Hirsimäki 16 Feb '05

16 Feb '05

The top level README file has a short C++ example of using the library. It contains three bugs: * Must use "f->" instead of "f." everywhere * f->setInputAlphabet() wants ConstAlphabetRef instead of (StaticAlphabet*) * It seems that setId() must be called for each state manually. Here is the correct (hopefully) version of the code: using namespace Fsa; StaticAutomaton *f = new StaticAutomaton; f->setType(TypeAcceptor); f->setSemiring(TropicalSemiring); StaticAlphabet *a = new StaticAlphabet(); f->setInputAlphabet(ConstAlphabetRef(a)); State *sp1 = new State(), *sp2 = new State(StateTagFinal); sp1->setId(0); sp2->setId(1); sp1->newArc(sp2->id(), f->semiring()->one(), a->addSymbol("a")); sp2->newArc(sp2->id(), f->semiring()->one(), a->addSymbol("a")); f->setState(sp1); f->setState(sp2); f->setInitialStateId(sp1->id()); ConstAutomatonRef fr = ConstAutomatonRef(f); -- Teemu Hirsimäki

2 2