Copyright © 1999-2011 LTU Technologies
|Revision $Revision: 1.113 $|
This guide is intended to system administrators and applies to LTU engine server 5.0
This guide is intended to system administrators and applies to LTU engine server 5.0.
The release notes for LTU engine server 5.0 can be downloaded from the LTU Technologies download center.
|Signature||Binary encoding of visual features (texture, color, shape, etc.).|
|Image DNA||See signature. Please note that "image DNA" and "image signature" are synonyms. "Image DNA" is used in general documentation, while "Image Signature" is used in technical documentation.|
|Kimage||The binary object attached to each image in the system. It contains a unique identifier, the signature, the MD5 and the reference to access the original and thumbnail images. It may also contains the keywords and the IPTC or EXIF data if any.|
|Repository||Every kimage in the system is assigned to a repository. A visual search is done in one, and only one, repository.|
|CBIR||Content-based image retrieval|
|GUI||Graphical User Interface|
|SOAP||Simple Object Access Protocol|
AutoScan client is only available for Windows users.
Please contact LTU Professional Services to obtain the Linux package.
PostgreSQL is Copyright © 1996-2008 by the PostgreSQL Global Development Group and is distributed under the terms of the license of the University of California
Java, J2RE, J2SE, and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.
Red Hat is a registered trademark of Red Hat, Inc.
Digital content has become a vital enterprise asset, and visual content management has emerged as a strategic necessity. LTU provides the advanced technologies and solutions that address this new imperative, enabling companies to manage and use visual assets more effectively.
Based upon LTU's innovative image recognition technology, LTU engine server simplifies the search process by enabling more intuitive navigation. It allows users to perform visual searches by images (instead of by name, keyword, number or description).
LTU engine server provides businesses with a competitive advantage by both cutting internal costs for database search and creating new revenue opportunities by making the user experience more efficient.
LTU products are based upon the same technological core: the LTU engine. The LTU engine bridges the gap between "low level", i.e. graphical, pixel related information, and "high level", i.e. semantic, user-friendly information.
To achieve this, the LTU engine computes a visual signature for every image that describe its visual content in terms of color, shape, texture and many higher order visual features. These descriptors are also called image DNAs or signatures.
By comparing and analyzing these signatures, the LTU engine can:
Find image clones
Find images that are visually similar
These functionalities, embedded in all of LTU's products, give businesses the ability to simplify indexing as well as search and retrieval processes within large image databases.
Please contact firstname.lastname@example.org for more information about the LTU engine.
The core data for LTU engine server is the signature, not the image itself. Once the signature is computed for an image, LTU engine server can, but does not need to store the image.
LTU engine server is a server-side software that provides businesses with the ability to search and navigate their visual assets.
The main features of LTU engine server are:
Visual navigation: Similarity search within a database to find a set of duplicate, clone or similar images.
Image upload: User submission of an image (any format, any source, still image, video or drawing) from its hard drive, real-time indexing and search for visually similar images.
Meta-data storage: LTU engine server provides the user with the ability to store information related to every image in its database.
Meta-data search: User can search the meta-data for an exact match retrieving the corresponding images.
Combined meta-data and visual search: The user can combine meta-data search and image retrieval.
Combined semantic/visual search: LTU engine server allows the user to attach and store keywords along with every image. The user can then search for images using keywords only, or perform very efficient search combining visual and semantic queries.
This section focuses on LTU engine server's server side software.
Designed to scale to very large systems and provide high availability, LTU engine server is a set of modular and distributable components. Each set of component is instantiated inside a component server, called a module. The following sections describe each module in detail.
The standard installation of LTU engine server (standalone installation) is hosted on a single machine, with only one instance of the WEKI module. LTU recommends such an installation for most common systems.
For very large systems, LTU recommends the distributed installation of LTU engine server. In order to take advantage of the CPU power of several computers, some modules can be distributed over various machines. To provide redundancy and high availability, one component can be duplicated over several machines.
Most systems use the standalone installation of LTU engine server. Distributed architecture is usually installed with the help of the LTU consulting group.
Figure 2.2, “Generic Distributed Architecture” summarizes a possible multi-component, distributed LTU engine server system. In a distributed system the roles of each module are:
WEKI: Handle HTTP requests, execute scenarios and compute signatures
KIMA: cache signatures, perform signature comparisons
The KIMA module stores signatures in RAM and performs signature comparisons. It is a sort of optimized database cache for image retrieval. In the KIMA modules, the signatures are hosted into repositories.
Signature repositories can be distributed over several KIMA modules. Thus, deploying a large number of KIMA modules makes it possible for the system to deal with very large signature repositories. The KIMA RAM footprint is linearly dependant on the number of signatures in the repository.
On a standalone installation, the KIMA is integrated in the WEKI module. When KIMA is integrated in the WEKI module, not all the features are availables.
If multiple KIMA modules are installed, each one of them has a unique identification number ranging from 1 to n where n is the total number of KIMA modules. This ID, defined by the administrator, is used to distribute repositories among all existing KIMA modules. Indeed, each repository has a unique internal number automatically assigned by the system.
One KIMA can load one or more repositories.
Example for 9 repositories with internal number from 1 to 9 and 2 KIMA modules: KIMA #1 will load repositories: 1, 3, 5, 7 and 9, while KIMA #2 will load repositories 2, 4, 6 and 8.
Partitioning one repository on several KIMA modules, hosted by several machines, has several advantages:
Improving the maximum number of signatures in that repository.
Reducing the image retrieval response time: signature comparisons are performed in parallel on all KIMA modules.
The partitioning is an override mechanism of the automatic repository deployment explained above and is based on the KIMA ID.
The WEKI module has three missions: handle HTTP requests, execute API requests, and compute signatures.
Since WEKI computes signatures, it is responsible for creating and deleting signature files on disk. As signatures are shared with the KIMA modules, KIMA and WEKI modules must have access to a shared storage area, if LTU engine server is distributed over several machines.
Signature computation is CPU intensive. It is necessary to compute signatures for image enrollment into the system or for image search by image upload. The total number of enrollments or searches by upload in a day is thus limited by the CPU power of the system. In order to enhance this power, one can run several WEKI modules on separate machines, and load balance the queries among these modules. There is no limitation on the number of WEKI modules that may be run in an LTU engine server system.
If using several WEKI modules, the client side load-balancing process that distributes queries among the different WEKI modules has to take session information into account.
When configuring the system, the LTU engine server administrator has to choose the best signature for his images. Indeed, bodies of images can have particularities that should be used to optimize search and navigation in the database.
Please choose the signature properly. If a misconfiguration were made, any previously added image would have to be re-computed.
LTU engine server analyzes all images at the same size. The bigger this dimension, the longer the CPU time. In order to reduce CPU time, the user can choose to use a smaller processing size, e.g. 192, that will reduce the signature computation CPU time by a factor of 2. Before using smaller signatures, the LTU engine server administrator should check that the quality does not downgrade during retrieval for a specific corpus.
Table 2.1, “LTU engine server signatures” summarizes the various signatures provided by LTU engine server.
Table 2.1. LTU engine server signatures
|Signature Id||Signature type||Working image dimension||Size (Bytes)|
|8||dominant colors signature||128x128||290|
|60 (60.1.0)||local matching||256x256||2056|
|60.1.1||similar to 60.1.0 , used for fineImageComparison||256x256||2056|
|62 (62.1.0)||local matching with rotation invariance||256x256||2056|
|65 (65.0.0)||local matching DNA 2010 with text removal and vertical flip support||256x256||2056|
|70 (70.1.0)||local matching DNA 2010 with asymetric retrieval||512x512||4096|
A signature type is configured for each repository.
To use fineImageComparison ( signature 60.1.1 ) , original images must be stored.
LTU engine server needs to have access to a database to start up. LTU engine server 5.0 supports 2 database softwares : Microsoft MsSql and PostGreSql.
LTU engine server is delivered with scripts for the initialization and administration of the database. You only have to create an user and a database for LTU engine.
If you have to update Microsoft SQL on a LTU engine server anterior to 3.0, please contact LTU support.
IMPORTANT : On linux, using default database settings, the PostGreSQL configuration is automaticly done by the installation script. If you don't need a special configuration, you can skip all this chapter.
On linux, LTU engine server requires PostGreSql 8.1 or higher (However, PostGreSql 8.4 or higher is recommended). Please refer to Table 4.3, “Required RPMs on Linux systems” to select linux postgresql related packages to install.
On windows, we recommend PostGreSql 9.0.
Database speed greatly depends on the internal index quality. We recommend the user of the PostGreSql command "vacuum" to recalculate indexes once in a while.
You must be logged as root to perform all the installation tasks.
Initialize databases and configuration files with a specific charset
When PostGreSql is run for the first time, it initializes the database calling "initdb". If you need the database to support another charset than the default one, you need to call "initdb" first with the right arguments. If you do not need support for special characters, you can jump directly to the next step.
The initialization must be done like this:
$ su - postgres $ initdb --encoding ISO8859-1
ISO8859-1" can be replace by the charset you need.
Contact LTU support for more information about database charsets depending on your PostGreSql version.
Authorize TCP access on PostGreSql
If you are using PostgreSQL version 8.x or newer, use the following instructions.
You need to edit PostgreSQL configuration file /var/lib/pgsql/data/postgresql.conf (or /etc/postgresql/<version>/main/postgresql.conf , /etc/postgresql/8.4/main/postgresql.conf for PostgreSQL 8.4).
Find configuration line that read as follow :
Next, set IP address(es) to listen on. You can comma-separated list of addresses. Default is 'localhost', and '*' is all.
With other version of PostGreSql
If neither of the above method is possible, contact LTU Technologies support.
$ /etc/init.d/postgresql start
This step is important to create the configuration file below.
Authorize the server to access PostGreSql
LTU engine server needs to access to PostGreSql via TCP connections, you need to explicitly authorize these connections. Edit the file '
/var/lib/pgsql/data/pg_hba.conf' (/etc/postgresql/8.3/main/pg_hba.conf for posgresql 8.3 and higher on Debian distribution) and add the authorize line at the end. For instance, to authorize the server "
10.0.0.56" to access to PostGreSql, add the following line:
host all all 10.0.0.56/32 trust
The general structure is :
host Database User CIDR-ADDRESS Method
host Database User IP-address IP-mask Method
For more details, refer you to http://developer.postgresql.org/pgdocs/postgres/auth-pg-hba-conf.html.
$ /etc/init.d/postgresql restart
Create the database user with database creation right
To create the user "
dbuser", use the command "
createuser". The following line creates the user.
$ su - postgres $ createuser -P -A -d dbuser
You have to give the user the rights to create databases.
Create the database instance
To create the database instance "
dbname", use the command "
createdb". The following line creates the db (assuming that the postgresql server IP address is 127.0.0.1).
$ createdb -h 127.0.0.1 -U dbuser dbname
$ createdb -h 127.0.0.1 -U my_ltu_dbuser my_ltu_dbname
The software is provided on our Download Center or in one compressed file named
This version does not support automatic upgrade from previous versions. Nevertheless you can use the database from previous versions (see Section 9.3, “How to use a new LTU engine server version with previous database”).
Prior to installation, make sure that a previous version of LTU engine server or Image Seeker is not installed.
This section provides a list of minimum software and hardware requirements for running LTU engine server 5.0.
Table 4.1. LTU engine server system requirements
|Operating Systems||CentOS 5, Debian 5, Debian 6, Windows server 2003 and Windows server 2008. Support by default 64bits Operating systems.|
|Software disk space||4 Gbytes on Linux, 8 Gbytes on Windows.|
|Signature disk space||The necessary disk space depends on the signature used and the number of images.|
|Thumbnail disk space||The thumbnail default size is 128x128. These images are around 5 Kbytes (it is possible not to store thumbnails).|
|Original image disk space||The original image size can dramatically vary with their dimension and their format (it is possible not to store original images).|
|Memory||2 Gbytes + necessary memory for signatures.|
|CPU on Intel arch.||Minimum: Intel Pentium IV 2GHz or equivalent. Recommended: Intel Pentium IV 3GHz or equivalent.|
|Database||PostGreSql 8.1 or higher ( 8.4 or higher recommended ), Microsoft SQL Server 2005.|
Signatures, binary, original and thumbnail images storage location is configurable. For the default image databases, the storage location is done on disk in partition "
/var" on Linux or c:/LTU/var. The disk space required depends on the number and the average size of the images, but to give an example, with photo images found on Internet, 4000 images with signatures and thumbnails takes 650 Mbytes on disk.
For LTU engine server distributed architecture, machine sizing depends on how many modules are running and the number of images hosted, in particular:
The KIMA modules load signatures in memory. Refer to the previous chapter to size the memory for each host running a KIMA module.
The WEKI modules handle HTTP requests and index images. They are more CPU consuming.
Regarding virtualized environment, we support it but only ones running under VM Ware.
The following table shows the list of databases supported by LTU engine server on Windows and Linux with 32 bits and 64 bits for standalone and distributed installation.
Table 4.2. Supported Databases on Linux and Windows
|OS||Standalone system||Distributed system|
|Windows 32bits||Microsoft Sql Server, PostGreSql||Microsoft Sql Server, PostGreSql|
|Windows 64bits||Microsoft Sql Server, PostGreSql||Microsoft Sql Server, PostGreSql|
LTU engine server has been tested on standard "server" installation of Centos and Debian Linux. The exhaustive list of the required files and software packages is in Table 4.3, “Required RPMs on Linux systems”.
Table 4.3. Required RPMs on Linux systems
|Distribution and version||Required RPMs||Remarks|
|CentOS 5 and higher||Needed packages for the system to work|
|CentOS 5 and higher||For the extra perl tools|
|Debian 5 and higher||Needed packages for the system to work|
|Debian 5 and higher||For the extra perl tools|
|All Linux versions||If using the database PostGreSql|
|All Linux versions||unixODBC ( unixodbc-dev for Debian )||For distributed mode|
|All Linux versions||postgresql-odbc ( odbc-postgresql for Debian )||For distributed mode with PostGreSql|
yum ( aptitude for Debian ) installation system when installing these packages as they may depend on other packages.
if you install LTU engine on 64bits OS, make sure that the software packages "freetype.i386" and "libX11.i386" are installed on your machine. On 64 bits OS you may need to install the package redhat-lsb-3.1-12.3.EL.el5.centos.i386.
if you are testing LTU engine on Red Hat Enterprise Linux Server 5.X (not officially supported) , you have to update a symbolic link with "ln -s /lib/ld-linux.so.2 /lib/ld-lsb.so.3".
The first step in the LTU engine server installation process is to choose between a standalone and distributed installation of the software. The standalone installation is suitable for most applications, you should choose the distributed installation in these cases:
You need to load a large database of images on one machine.
You cannot load the whole database of images on one machine.
You want to split the image database over several machines to insure fail-over and load-balancing of your system.
The default parameters of LTU engine server standard distribution are set for distributed installation.
If LTU engine server is to be distributed on several servers, the same software distribution and configuration files must be installed on all servers. The configuration for each machine is different depending on the modules running. Distributed installation and configuration is detailed in Chapter 5, Deploying a distributed LTU engine server system.
Upgrading from previous version require backing-up the software configuration, the uninstallation of the previous version and the installation of the new version. The database content can be kept, the schema will be upgraded if necessary when installing the new version.
The database schema upgrade from previous versions is automatic and is done in the database manager tool "dbManager" which is run during the install or can be executed separately.
The tool detects that an upgrade is necessary and automatically upgrade the schema. The upgrade will not alter any of your data but just add column in tables or/and add new tables. The upgrade is done on all repositories present in the database.
Before starting the installation process, make sure you have fulfilled the following tasks.
Before running the installation script, you must first setup the database access where LTU data will be stored. Please refer to Chapter 3, Setting Up the Database.
The database configuration needed is:
On PostGreSql and MSSQL, you need a user with database creation rights.
The installation program will use these parameters to connect to the database and create the necessary schema.
The installation program will install the license you provide. Please refer to Section 8, “LTU engine server license” for more details about license information.
For an installation from an archive file downloaded from LTU's client download center, you have to uncompress the archive file in a temporary directory. The archive file contains necessary files for installing LTU engine server as well as standard LTU engine server plug-in files or patches if any.
If LTU engine server is already installed and you want just use a new license, you absolutely must remove old license from the folder "/etc/ltu".
Create a temporary directory
$ mkdir /tmp/LTUEngineServerInstall
Uncompress archive file in the temporary directory
$ cd /tmp/LTUEngineServerInstall
$ tar -zxvf LTU-Engine-Server-5.0-0086.ix86.linux.tgz
The installation and the configuration of LTU engine server are done with the program "
ltu-setup". Once the installation is done, the program may be run several times to configure or re-configure the software (the installation part will be skipped).
The default installation directory is "
/home/ltu" on Linux but it can be changed to any shared or local directory. In this section, the software is installed in "
To start the installation, run ltu-setup.
**************************************************************************** * * * LTU-Engine Server installation and setup * * * * Copyright (c) 1999-2011 LTU Technologies * * All rights reserved * * * **************************************************************************** -------------------------------------------------------------------- 1: Shell libraries -------------------------------------------------------------------- Loading LTU Technologies main shell library [ OK ] Loading linux shell library [ OK ] Loading database shell library [ OK ] Loading LTU Technologies install shell library [ OK ] Install log file: /tmp/ltu-setup.211204.log Refer to this file if you encounter any problem during the installation process.
The installation program generates a log file (in our example:
/tmp/ltu-setup.211204.log). The administrator can refer to this file for installation details and possible errors.
-------------------------------------------------------------------- 2: Operating system checks -------------------------------------------------------------------- System compatibility [ OK ] Read version file 'VERSION.txt' [ OK ] Check OS compatibility (VERSION.txt) [ OK ] Checking required packages [ OK ] Checking optional packages [ OK ] Checking optional package 'postgresql' [ OK ]
First; the system checks the operating system according to system and OS compatibility with LTU engine server 5.0 as well as according to required and optional packages.
--------------------------------------------------------------------- 3: Installation -------------------------------------------------------------------- STATUS: LTU engine server is not installed on your host Read version file 'VERSION.txt' [ OK ] -------------------------------------------------------- ------ ./VERSION.txt Build generation date : 2011-03-16 01:46:47 Product : LTU-Engine-Server 5.0 Build number : 0000 Target system(s) : CentOS 5.4 ix86, CentOS 5.5 ix86, CentOS 5.3 ix86, Debian 5.0.4 ix86, Debian 5.0.5 ix86, Debian 5.0.6 ix86, Debian 5.0.7 ix86, Debian 5.0.8 ix86, Debian lenny/sid ix86, Debian squeeze/sid ix86 -------------------------------------------------------- Do you want to install LTU engine server on this host (Y)es/(N)o? [Y]
Second, the program checks if LTU engine server is already installed on the system. If it is not, the program displays the information about the version you are about to install (file "
VERSION.txt" from the distribution).
When the installation program asks a question, it may display a default answer, to select this answer, type
ENTER. When possible responses are proposed, the only allowed answers (apart from the default value) are surrounded by parenthesis. For example, the yes/no question requires Y or N for answer.
The Linux operating system may be not be officially supported by LTU Technologies , in that case, the install program will stop indicating that the operating is not supported. To bypass this check, you may rerun ltu-setup with the option "
The install directory /home/ltu is where all LTU engine server binary files will be installed. As administrator, make sure that the directory is in a partition with enough disk space (see Table 4.1, “LTU engine server system requirements”).
-------------------------------------------------------------------- 3.1: LTU engine server installation -------------------------------------------------------------------- Creating directories Unpacking LTU-Engine-Server configuration files [ OK ] Unpacking LTU-Engine-Server binary files [ OK ] Setting installation directory in configuration file [ OK ] Copy ltud to /etc/init.d/ [ OK ] Copy mond to /etc/init.d/ [ OK ] Change startup script owner [ OK ] Register LTU engine server services [ OK ] Register mon service [ OK ] Use system apache for admi module [ OK ] Installation of LTU engine server on this host [ OK ]
LTU may have provided patches. In this case, they are installed at this step. In this example, no patch has been provided and thus no patch will be installed.
-------------------------------------------------------------------- 4: Patch installation --------------------------------------------------------------------
Plugin files if any can also be installed. In this example, no plugin file will be installed.
-------------------------------------------------------------------- 5: PlugIn installation --------------------------------------------------------------------
In case of a distributed installation, make sure that the ltu user id will be the same on all servers of the distributed system.
-------------------------------------------------------------------- 6: System check -------------------------------------------------------------------- Deleting group ltu [ OK ] Adding user ltu [ OK ] Link '/etc/ltu/perl-libs' [ OK ] Link '/etc/ltu/Smarty' [ OK ] Create log directory [ OK ] Create run directory [ OK ] Create fe directory [ OK ] Create fe/templates_c directory [ OK ] Create fe/views directory [ OK ] Create fe/datamodels directory [ OK ] Create fe/images directory [ OK ] Create fe/exports directory [ OK ] Do you want to install a license file (Y)es/(N)o? [Y]
The setup program checks the installation and enters into the license installation process. You may choose to skip the license installation and install it afterwards using "
ltu-setup" again or manually by copying the license file in "
/etc/ltu" (with ".lic" extension, please refer to Section 8, “LTU engine server license”).
You may install the license from hard drive (option directory). Once you have chosen a license, the installation program asks you to confirm. If you want to install another license at that moment you can answer "A" (for abort), the license installation process will restart from the beginning.
The program does not check the validity of the license. Please refer to Section 8, “LTU engine server license” for more information.
Installing your license Select a media < directory,skip > [directory]: directory
In this example, the licence file is located in the directory
Enter the directory [/tmp]: /home
File list in (/home) [ 0 ] : my_license.lic Choose a number : 0
The features of the licence will be displayed. You have then to confirm if you want to install the licence.
Install the license (A)bort/(C)ontinue? [C] C
Copying license file to /etc/ltu [ OK ] Status of Install license [ OK ] -------------------------------------------------------------------- Installation of LTU-Engine Server on this host [ OK ] --------------------------------------------------------------------
The software installation part is now finished. When running the "
ltu-setup" program again, this part will be skipped.
Now starts the LTU engine server configuration part.
-------------------------------------------------------------------- 7: LTU-Engine Server configuration -------------------------------------------------------------------- Do you want to configure LTU-Engine Server (Y)es/(N)o? [Y] Y
You may choose to exit the program now to configure your system manually. It is however recommended to use the installation setup for configuring LTU engine server, like in this example.
The host Internet address must be the fully qualified Internet name of the host. This name must be understandable by client hosts that will use LTU engine server (do not put addresses like "localhost" or "127.0.0.1").
The host internet address can be the host name or the IP address. It is used for HTTP references in LTU-Engine Server administration interface and application API and must therefore be accessible by client hosts. Enter this host internet address [myhost.mydomain.com]: myhost.mydomain.com
You may also choose at this moment to secure the access to web interface by SSL.
Setting the host internet address in global configuration file [ OK ] The web-based graphic user interface is not secured with SSL. Secure the web interface access with SSL (Y)es/(N)o? [N] N Disable SSL for web interface access [ OK ]
We configure the system to run a KIMA, a WEKI and an ADMI on this host. For more information about installing a distributed system see Chapter 5, Deploying a distributed LTU engine server system.
8: LTU-Engine Server distribution setup -------------------------------------------------------------------- -------------------------------------------------------------------- 8.1: Distribution configuration -------------------------------------------------------------------- The module KIMA is activated on this host Do you want to run a KIMA on this host (Y)es/(N)o? [Y] Enter KIMA Id : The module WEKI is activated on this host Do you want to run a WEKI on this host (Y)es/(N)o? [Y] The module ADMI is activated on this host Do you want to run a ADMI on this host (Y)es/(N)o? [Y]
We use default WEKI and/or KIMA services capacity.
-------------------------------------------------------------------- 9: LTU engine system capacity configuration -------------------------------------------------------------------- Checking allowed cpu capacity Do you want to configure WEKI and/or KIMA services capacity ? (Y)es/(N)o? [N]
License server is now activated.
-------------------------------------------------------------------- 10: LTU License server setup -------------------------------------------------------------------- License server activated
The program now enters into the database configuration process, by default, it creates a local PostgreSQL database instance.
If you answer yes to "edit the database parameters", and no to "use the default database storage", you have to provide the database parameters (the database name, its address, its port, the login name and the login password). When running the installation program the first time, the default values start with "EDIT::..." for editing convenience otherwise the program shows you the current database settings and asks if you want to change them.
-------------------------------------------------------------------- 11: LTU engine server database setup -------------------------------------------------------------------- -- Default database storage with the following parameters: - Database user login : ltu_user - Database user password : ltu_password Do you want to edit the database parameters (Y)es/(N)o? [N] Y Do you want to use the default database storage (Y)es/(N)o? [Y] N Database module in global configuration file [ OK ] Enter the database address [127.0.0.1]: Enter the database instance name [ltu_engine]: Enter the database port : Enter the database login name [ltu_user]: Enter the database login password [ltu_password]: -- Current Database Parameters - Type : POSTGRESQL - Database address : 127.0.0.1 - Database name : ltu_engine - Database port : 5432 - Database user login : ltu_user - Database user password : ltu_password Confirm parameters (A)bort/(C)ontinue? [C] Checking postgresql client binaries [ OK ] Database access (Version: 8.1.22) [ OK ] Checking required packages [ OK ] Change ODBC PostGreSql driver [ OK ]
After you have confirmed the database parameters, the installation program checks that it can connect to the database. In case, the installation program fails to connect to the database you be asked to enter again the database parameters or to exit the installation process.
After the connection to the database is checked, the installation runs the console tool 'db-manager' that will help install the right schema template corresponding to your needs. The administrator may run this tool at any time after installation (c.f. Chapter 6, Administrating LTU engine server).
When the database schema is empty, the tool creates the common tables for all image databases (i.e. repositories).
Launching image database manager... ****************************************************************************** * Image Database Manager - Version 2.0 * * LTU Technologies - Copyright (c) 1999-2010 * * All rights reserved * ****************************************************************************** checking database content... ok adding/updating host configuration... ok initializing the image database status... from the database from the front-end configuration from the database sql files updating the status of all databases initializing the repository skeleton list... ok initializing the signatures configurations list... ok Change current partition string '', enter a new partition string or 'cancel': new partition string to ''. (A)bort/(C)ontinue? [C]
The program shows you the available image repositories and their status on the system. When installing for the first time, the program shows an empty list of image repositories and the actions the administrator can do. In the fisrt time, Enter '1' to create a new repository.
Id : repository identifier
Desc : repository description
Fe : front-end, can take the value 'Y' (actived) or 'N' (desactived)
Db : Database
Ba : backend, can take the value 'Y' (actived) or 'N' (desactived)
Nb Img : number of images stored in the repository
Signature : identifier of the signature used
====================== Available image repositories ====================== Id Desc Fe Db Ba Nb Img Signature -- ---- -- -- -- ------ --------- * Choices 0/ Quit 1/ Create a new repository 2/ Select/Unselect one repository 3/ Enable selected repositories in frontend and backend 4/ Disable selected repositories in frontend and backend 5/ Uninstall selected repositories in database and in frontend 6/ [ADVANCED CONFIGURATION] Change the signature type of the selected repositories 7/ [ADVANCED CONFIGURATION] Change the search signature type of the selected repositories 8/ Change the repository partition string '' Your choice ? 1
In the screen below, the administrator can choose between seven repositories templates.
After selecting a repository template, you must complete all parameters related to this new repository (ID, repository name,....).
========== Available repository templates ========== ID Name -- ---- 1 Color Query 2 Matching - Fine Image Comparison 3 Matching - Sig60 - does not support rotation 4 Matching - Sig62 - supports rotation 5 Matching - Sig65 - media monitoring 6 Matching - Sig70 - asymetric retrieval 7 Similarity - Sig02 Choose a repository template : 3 Choose your repository ID : Choose your repository name [rep1]: repository1 Choose your repository title in English [Repository 1]: english_title Choose your repository title in French [Repository 1]: titre_francais Choose your repository description in English : english_description Choose your repository description in French : description_francaise *** Your duplication parameters : Repository templates = Matching - Sig60 - does not support rotation Your repository name = repository1 Your repository id = 1 Your repository English title = english_title Your repository French title = titre_francais Your repository English description = english_description Your repository French description = description_francaise Are your parameters OK ? (A)bort/(C)ontinue? [C]
Now that you have entered all parameters, you can enter 'yes' to install the image database.
/etc/ltu/admi/skeletons/Matching - Sig60 - does not support rotation/admi/databases/_REPNAME_ /etc/ltu/admi/databases /etc/ltu/admi/skeletons/Matching - Sig60 - does not support rotation/sql/build/postgresql/rep_ID_ /home/ltu/tools/database/build/postgresql initializing the image database status... from the database from the front-end configuration from the database sql files updating the status of all databases... ok installing image database with Id '1'... create database tables and insert default values... run sql file 'rep1_ltu_tables.sql' run sql file 'rep1_ltu_values.sql' run sql file 'rep1_common_entity_tables.sql' run sql file 'rep1_entity_tables.sql' run sql file 'rep1_common_entity_values.sql' ok enabling image database with Id '1' in frontend... ok creating binary directory '/var/ltu/rep1/binary/'... ok creating image directory '/var/ltu/rep1/images/'... ok WARNING: you need to restart manually the services. [Press Return to continue] ====================== Available image repositories ====================== Id Desc Fe Db Ba Nb Img Signature -- ---- -- -- -- ------ --------- ( ) 1 english_title Y Ok Y 0 60 * Choices 0/ Quit 1/ Create a new repository 2/ Select/Unselect one repository 3/ Enable selected repositories in frontend and backend 4/ Disable selected repositories in frontend and backend 5/ Uninstall selected repositories in database and in frontend 6/ [ADVANCED CONFIGURATION] Change the signature type of the selected repositories 7/ [ADVANCED CONFIGURATION] Change the search signature type of the selected repositories Your choice ? 0 exiting...
During post installation, scripts can be included in the installation procedure in order to finalize the configuration of a plugin.
-------------------------------------------------------------------- 12: Post Installation --------------------------------------------------------------------
The installation program finally tries to start LTU engine server and verifies if all modules are running.
-------------------------------------------------------------------- 13: Installation finished -------------------------------------------------------------------- Do you want to start LTU-Engine Server (Y)es/(N)o? [Y] Y Restarting LTU-Engine Server [ OK ] Status LTU-Engine Server [ OK ] -------------------------------------------------------------------- STATUS: HOST your_hostname INSTALLATION ENDED SUCCESSFULLY
LTU engine server is now running on your system. The next step is to configure the Internet based interface to give access to content administration users and search users. Section 4, “Web-based administration interface” explains you how to do so.
After LTU engine server installation, you can configure syslog logging for kima, weki and jni.
To activate syslog logging for kima, update logger.syslog.bEnable = False to logger.syslog.bEnable = True in /etc/ltu/kima-logging.conf.
To activate syslog logging for weki, uncomment #handlers= com.ltu.utils.logging.SyslogHandler in /etc/ltu/weki.logging.properties.
To activate syslog logging for jni, update logger.syslog.bEnable = False to logger.syslog.bEnable = True in /etc/ltu/ltujni-logging.conf.
You can configure extra syslog parameters in the file listed above as standard syslog usage. For more information, please consult syslog documentation.
Note : by default, the syslog messages will be found in /var/log/messages.
Note: If using the database MS SQL Server 2005, please install SQL Server Service Pack 2 on the windows machine.
LTU engine server has been tested on Windows Server 2003 and Windows Server 2008. LTU engine server does not require any third party software.
LTU engine server requires access to a database. The database user must have rights to create tables in the database. Thus, before running the installation script, ensure that a database has been set up.
LTU engine server 5.0 supports 2 types of database software: PostGreSql 9.0 and Microsoft SQL Server 2005. Prior to installation, gather the following information about the database:
Type of database (PostGreSql or MsSql)
Address of the database server
Name of the database instance
Port of the database server: confirm the default values suggested by the installation setup (step 5)
Name of the database user
Password of the user
The default installation directory is "c:\LTU" on Windows but it can be changed to any local directory. In this section, the software is installed in "c:\LTU".
If LTU engine server is already installed and you want just use a new license, you absolutely must remove old license from the folder "c:\LTU\etc".
Unzip the archive
LTU-Engine-Server-5.0-0086.ix86.win.zip into the directory c:\ .
On Windows 64-bits run
c:/LTU/tools/install/vcredist_x64.exe. If you are running a 32-bits Windows run
Reboot the machine
Copy the specific following libraries "msvcp90.dll", "msvcr90.dll" and "mfc90.dll" from c:\LTU\ltuSystem\native\win32 to c:\WINDOWS\system32. This step may not be necessary depending on the machine configuration.
Install the .NET Framework 3.5 Service pack 1 from c:\LTU\tools\install (for Windows server 2008, use Server Manager/Features/Add Features/.NET framework 3.5.1).
Copy the LTU engine server license file into
The license file name should have the extension ".lic". Please refer to Section 8, “LTU engine server license” for more details about license information.
c:/LTU/etc/ltu/global.bat and change the following options:
|Parameter name||Parameter content|
|DATABASE||Type of database used: MSSQL or POSTGRESQL|
|DATABASE_ADDR||IP address of the database server|
|DATABASE_NAME||Database schema name|
|DATABASE_PORT||Database server listening port. Default values are 5432 for Postgresql and 1433 for MsSql.|
|DATABASE_LOGINNAME||Database user login name|
|DATABASE_LOGINPWD||Database user password|
|LMHOSTID||MAC address of the machine without the "-" characters (e.g. 00B0D03BE90E)|
|KIMA_64||Use 64bits Kima (1 by default, set to 0 to use 32bits Kima)|
You may also increase the amount of memory allowed by the signature cache module (WEKI). To increase the amount of memory, simply increase the value of the parameter WEKI_MEM_SIZE. The default value is 800 MB. This value should not be higher than 2048 MB.
Kima module access to the database with ODBC system, to install and configure ODBC, please refer to the windows Section 3.1, “Configure ODBC access to the database”.
In a command prompt window, change directory to c:\LTU\tools\install and run db-manager.bat
1/ Create a new repository
2/ Selected/Unselected one repository
3/ Enable selected repositories in front-end and backend
4/ Disable selected repositories in front-end and backend
5/ Uninstall selected repositories in database and in front-en
Db-manager tool is used in the same way as on Linux. Please refer to Section 5.2 for more details about using db-manager.
LTU engine server is now installed on your machine. You should now manually start the LTU engine server modules to check that the server is correctly configured. Open a new command line console window for each.
CHECKING FLXM CONFIGURATION To check is flxm is configured correclty, run the following
For more information, please see Section 8, “LTU engine server license”.
CHECKING ADMI CONFIGURATION To check if admi is configured correctly, run the following:
If successful the command will say '+ Run Apache 2' and not return or show errors. Leave this window open and proceed to the next test.
CHECKING WEKI CONFIGURATION To check if the weki is configured correctly, run the following:
If successful the window will scroll pages of log messages and complete by returning with no errors to the command line.
RUNNING THE LTU ENGINE SERVER One you have verified that admi and weki run and are installed, you are now ready to start LTU engine server.
START VIA CONSOLE To run LTU engine server via the console will open two windows. The server will run for as long as these two windows are open, closing the windows will stop the server. Run the following:
START VIA WINDOWS SERVICE To install LTU engine server as a Windows service. You only need to be install the services once. You may have problems if you try to install services when they already exist and are running. After a successful install there will be four new services 'LTUADMI', 'LTUWEKI', 'LTUFLXM' and 'LTUWEKI'. To install run the following:
This script should install the four Windows services named "LTUADMI (Apache 2)", "LTUKIMA", "LTUFLXM" and "LTUWEKI (Tomcat server 5)".
UNINSTALL THE LTU WINDOWS SERVICES To uninstall the services, run the following:
For log rotation use log-kima-rotation.bat that you have to add in a new scheduled task (every month for example). Logs rotated with 10 logs files.
LTU engine server software is protected by a license. Every machine that is part of the LTU engine server system must have a license. Ask your LTU software reseller for a valid license or ask LTU Technologies directly by sending an email to license-request@LTUtech.com.
In the case of a missing or invalid license file, LTU engine server License Server fails to start. This is the error you will get on Linux:
$ /etc/init.d/ltud status License Server (23129) [ FAILED ] KImage Repository Module (23160) [ OK ] Web KImager Module (23487) [ OK ] ADMInistration Module (23501) [ OK ]
The license server is activated by default. It is defined on linux by the FLXM_RUN in /etc/ltu/local.conf and on windows by FLXM_RUN in "[c:/LTU/]etc/ltu/global.bat"
For each machine that is a part of your LTU engine server system, you will need to have a valid license. On Linux and Windows, get the MAC-Ethernet address, the number of CPUs and the hostname.
If LTU engine server is already installed, you can use a local program to directly get the host Identifier information:
$ [LTU_HOME]/bin/linuxCO46/lmhostid Lmhostid - Copyright (C) 1989-2001 Globetrotter Software, Inc. The FLEXlm host ID of this machine is "00B0D0384919"
On Linux, to obtain the MAC address of the primary network card:
$ ifconfig -a eth0 Link encap:Ethernet HWaddr 00:B0:D0:38:49:19 inet addr:10.1.10.24 Bcast:10.1.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:54939 errors:0 dropped:0 overruns:1 frame:0 TX packets:21471 errors:0 dropped:0 overruns:0 carrier:1 collisions:0 txqueuelen:100 Interrupt:5 Base address:0xec00 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:114 errors:0 dropped:0 overruns:0 frame:0 TX packets:114 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0
The MAC address is "00:B0:D0:38:49:19".
To obtain the MAC address of your machine:
Open a command prompt in Windows
All information about your network connection is displayed, including the physical address of your network interface (the MAC address):
Physical address : 00-04-76-D8-30-BB
Provide this MAC address to LTU technologies to obtain a valid LTU engine server license.
On Linux, to obtain the number of CPU:
$ grep processor /proc/cpuinfo | wc -l 2
On windows, to obtain the number of CPU, you will need to open the machine property window.
Only one FLXM module must be started for an LTU engine server system (see Chapter 7, Configuring LTU engine server on how to start the license server). This process (called lmgrd located in the bin directory) is a lightweight process using almost no resource. By default it uses the port 27000.
Get all requested information from the unique host and ask your LTU software reseller for a valid license or ask LTU Technologies directly by sending an email to license-request@LTUtech.com.
The license file has a ".lic" extension and have to be placed in "/etc/ltu/". In case of a multi host installation, the license file must be only installed on the FLXM module host.
Do not move/remove the license file, the software checks it periodically.
Format of a simple license file:
SERVER MY_SERVER 005056B50014 VENDOR ltutech FEATURE 60 ltutech 1.000 31-jan-2012 uncounted HOSTID=005056b50014 \ TS_OK SIGN="0087 973F 44F0 CDD8 97AE 8C5A 0E85 E300 A2AC BAE2 \ A490 E320 FBEB F75A 7BF9" FEATURE indexation ltutech 1.000 31-jan-2012 uncounted \ HOSTID=005056b50014 TS_OK SIGN="002E 60FF B711 0D24 015B 23D0 \ 8A09 3000 5A83 F74A 97BC 09DB CE1B 8B06 611A" FEATURE recognition ltutech 1.000 31-jan-2012 uncounted \ HOSTID=005056b50014 TS_OK SIGN="0084 C4F2 0CDF BC10 561A 9930 \ FEEB 2D00 5DC4 DF52 5E05 5320 16F5 5E25 AF5F" FEATURE retrieval ltutech 1.000 31-jan-2012 uncounted \ HOSTID=005056b50014 TS_OK SIGN="00B7 9FD4 707A 37ED CB20 D6AA \ 7BEE AD00 866D 1CB4 EE65 306D 1432 E058 1A3F" FEATURE ltu-kima-server ltutech 1.000 31-jan-2012 uncounted \ HOSTID=005056b50014 TS_OK SIGN="0078 CDEF A771 3296 1418 C390 \ 0A42 8A00 924D 2720 CEE5 0E5E B951 322F CB14" FEATURE ltu-cpu-capacity ltutech 1.000 31-jan-2012 2 \ HOSTID=005056b50014 TS_OK SIGN="0088 71BF 565E 879C 6C40 B120 \ 43A2 3000 D37F 0179 BEF9 2F07 DC9F F721 9589" ### LTU engine server: Database limitation #### 10k images FEATURE db-nbimages-add-limit ltutech 1.000 31-jan-2012 10000 \ HOSTID=005056b50014 TS_OK SIGN="004F 04BA 3113 9979 92BD 8EB7 \ 968F F400 8B9F 41C0 4922 2855 A3A1 DA8E DD64" FEATURE db-nbimages-system-limit ltutech 1.000 31-jan-2012 10200 \ HOSTID=005056b50014 TS_OK SIGN="0046 87D0 3B63 B713 1B89 F379 \ E487 E400 CAC2 D600 82A7 676D C9D2 C55A 49E8"
ltu-cpu-capacity" is the number of CPU allowed on the global system. In case the number of CPU is not limited, this value is "uncounted". The license contains several features (each is a line starting with the word FEATURE). The features contain what LTU engine server can do. The main features are: "indexation" and "retrieval" and the signature identifier allowed.
All modules check the license server for a valid license at startup. Following are log message examples (c:\LTU\var\log\ltu\flxm.log by default) :
license manager: can't initialize:Cannot find license file. The license files (or license server system network addresses) attempted are listed below. Use LM_LICENSE_FILE to use a different license file, or contact your software provider for a license file. Filename: "/etc/ltu/*.lic" License Path: "/etc/ltu/*.lic" FLEXnet Licensing error:-1,359 System Error:2 No such file or directory For further information, refer to the FLEXnet Licensing documentation,available at "www.acresso.com". Using license file "/etc/ltu/*.lic"
Std out log: Mar 10 10:03:23 ERROR Invalid (inconsistent) license key The license-key and data for the feature do not match. This usually happens when a license file has been altered Feature: ltu_feature_1 License path: /etc/ltu/my_license.lic
Std out log: Mar 10 13:13:30 INFO Checking license
Please do not hesitate to contact our support team if you face any issue or if you have any question regarding procedures described in this chapter.
First you need to backup your database, if you are using the default one please follow the procedure, please replace name in capital letter by your parameter.
Connect to a console and use this command:
go to the directory where you want to backup
pg_dump -U ltu_user ltu_engine --file backup_ltu_engine_db.dump --clean
cd "C:\Program Files\PostgreSQL\9.0\bin"
pg_dump.exe -f c:\YOUR_BACKUP_DIRECTORY\backup_ltu_engine_db.dump --clean -U YOUR_USERNAME DATABASE_NAME
Then backup your directories:
Unpack your archive with the var/ltu and etc/ltu folders in / for linux and in c:\LTU for windows.
Connect to a console and use this command:
psql -U ltu_user -d ltu_engine -f /YOUR_BACKUP_DIRECTORY/backup_ltu_engine_db.dump
cd "C:\Program Files\PostgreSQL\9.0\bin"
psql.exe -U YOUR_USERNAME DATABASE_NAME <c:\YOUR_BACKUP_DIRECTORY\backup_ltu_engine_db.dump
To use an existing LTU engine server database with a new LTU engine server version, you have to :
Backup your data (see Section 9.1, “How to backup LTU engine server”)
Reinstall the new version (see Section 6.2, “Installing and configuring LTU engine server”)
Restore the data (see Section 9.2, “How to restore LTU engine server”)
Any LTU engine server system can be distributed over several hosts, so as to build a highly available, scalable system. This section describes the step-by-step "how to" deploy distributed LTU engine server. See Chapter 2, LTU engine server Overview for a general overview of the distributed LTU engine server system.
Distributed LTU engine server requires 3 kinds of modules:
One or more KIMA modules
One or more WEKI modules
One or more ADMI modules
Many ADMI modules can be installed on several machines to build a load balanced / fail over web configuration. In order to be able to share correctly the data between the ADMI modules, this configuration requires an NFS file server to be set up.
There are many ways to achieve a load balancing/fail over solution for web servers. Hardware will be the most reliable, software may be less cost-consuming. There are only a few rules to configure ADMI modules so that they can share data with other ADMI, they are in the following chapter.
They use CPU power quite a lot, but do not require a lot of RAM. Deploying a large number of load-balanced WEKI modules will enable a great number of client to do queries simultaneously.
They require fast and large RAM and CPU power.
The number of signatures hosted by a KIMA module is limited by the available RAM size. Since 4Gb is the maximum RAM that can handle a process on a 32bits machine, you should use a 64bits operating system to overpass this limit.
It is recommended to refer to LTU consulting team to validate your distributed LTU engine server system architecture.
Designing the architecture means choosing what modules to install on what host. The constraints on the architecture design are the following:
There are as many WEKIs and KIMAs as possible
The total amount of memory allocated to KIMA modules is enough to cache all the signatures of all the repositories
One machine can host one instance of WEKI and KIMA modules simultaneously. One server cannot host several instances of the same module.
You need to deploy and install LTU engine server on all the hosts of the architecture. To do so, on every host, follow the instruction from Chapter 4, Installing or upgrading LTU engine server, and enter the distribution configuration part.
Now, you have to configure every host according to the architecture design elaborated above. This section describes how to configure each host step by step. In the installation detailed above, we are installing a host with all components on a machine and another machine with only a KIMA running. Each repository will be hosted on each of the two KIMA modules according to its identification number.
See Chapter 7, Configuring LTU engine server for details on all parameters that are used in this section.
If you plan to store the images and signatures on disk. You should make sure that the 'ltu' user created on every machine of the system has the same Id so that all module can access the files.
Kima module access to the database with ODBC system. Using default installation settings, you don't need to manually configure postgresql odbc. Using specific configuration, you will need to install an ODBC client and ODBC drivers. On linux, unixODBC (http://www.unixodbc.org/) manage PostGreSql database. A package called unixODBC is available for redhat distributions ( CentOS ). See Table 4.3, “Required RPMs on Linux systems” for more information on required odbc packages.
After installing this package, follow those instructions :
Install the package called postgresql-odbc and then edit those files :
[PostgreSQL] Description = ODBC for PostgreSQL Driver = /usr/lib/psqlodbc.so Setup = /usr/lib/libodbcpsqlS.so FileUsage = 1
[LTU] Description = PostgreSQL Driver = PostgreSQL Trace = No TraceFile = Database = EDIT::Your database name Servername = EDIT::Your server address Username = EDIT::Your login Password = EDIT::Your password Port = EDIT::Your port (by default : 5432) Protocol = 6.4 ReadOnly = No RowVersioning = No ShowSystemTables = No ShowOidColumn = No FakeOidIndex = No ConnSettings =
The KIMA module configuration through the installation process is described below. The corresponding configuration parameters are explained after.
-------------------------------------------------------------------- 8.2: Distribution configuration -------------------------------------------------------------------- The module KIMA is not activated on this host Do you want to run a KIMA on this host (Y)es/(N)o? [N] Y Enter KIMA Id : 1
Each KIMA server has an ID, that is set in "
/etc/ltu/local.conf". To set a KIMA's ID to N, make sure the KIMA_ID variable is set to N in this file:
KIMA IDs are integers between 1 and N, if N is the total number of KIMA modules in the system.
Some files must be shared by all ADMI modules : place them on the NFS server and then replace them on each ADMI server by symbolic links pointing on NFS server.
Note that the user "ltu" from group "ltu" created on each machine will have to share the same userid to be able to access in read/write mode on all shared files.
The files and directories to be shared are listed below :
VARS_DIR/users where VAR_DIR is the directory storing users data and images (/var/ltu by default)
INSTALL_DIR/ltuSystem/http_data/htdocs/tmp where INSTALL_DIR is the installation directory.
There may be two ways of configuring a load balancer application : either it will redirect randomly any request to one of the ADMI, either it can analyze the request header, get the PHP session id and depending on it redirect to the given ADMI that has generated this ID.
In that last case, once a user is logged on a given ADMI, he will always stay on it, and there is no need for the PHP servers to share their session data with other.
In the first case however, once a user is logged on an ADMI server, he can then be redirected during the same session on any ADMI : they have to share their session data with the originating server. Follow the steps below to achieve this :
Create a directory on the NFS server that can be accessed by all ADMI
/etc/ltu/admi/php.ini file of each ADMI server, change the property session.save_path to this shared directory.
All ADMI modules must be configured to point on the one and only WEKI module of the application. There are to location to change :
/etc/ltu/admi/databases/rep_name/repXXX_config.xml, set the "host" property of the SERVER element to the address of the WEKI server :
<SERVER apiVersion="4.0" host="WEKI_SERVER" logLevel="-1" port="8090" retry="0" timeout="0"/>
httpd_2.0.conf of each ADMI, change the proxy setting of Apache to redirect all LTU engine server API queries on the WEKI :
ProxyPass /iseeker/ http://WEKI_SERVER:8090/iseeker/ ProxyPass /ifilter/ http://WEKI_SERVER:8090/ifilter/ ProxyPass /common/ http://WEKI_SERVER:8090/common/
The following load balanced configuration was tested (see figure below). It does not implement fail-over, but was enough to test the sharing of data between many deployed ADMI module. Thus, using any solution (hardware or software) to implement a load balance and/or fail-over solution will work.
1 SQUID Server : squid is a common HTTP accelerator and proxy cache for web applications. It is used here to force the DNS client to ask the round robin every minute. That way; nearly every queries are directed on a different ADMI server.
1 DNS Round Robin : DNS server serving many I.P. addresses for one host name
1 NFS Server : The file sharing server
N ADMI's : many ADMI modules, sharing some data via the NFS server.
1 WEKI : in this version, it is not possible to use more than one WEKI module.
In the 8.1 configuration part, you can select which KIMA, WEKI and ADMI services you want to run on this host.
-------------------------------------------------------------------- 8.1: Distribution configuration -------------------------------------------------------------------- The module KIMA is activated on this host Do you want to run a KIMA on this host (Y)es/(N)o? [Y] Enter KIMA Id : The module WEKI is activated on this host Do you want to run a WEKI on this host (Y)es/(N)o? [Y] The module ADMI is activated on this host Do you want to run a ADMI on this host (Y)es/(N)o? [Y]
During LTU engine server database setup for a distributed installation, you can specify a partitioning string.
Launching image database manager... ****************************************************************************** * Image Database Manager - Version 2.0 * * LTU Technologies - Copyright (c) 1999-2010 * * All rights reserved * ****************************************************************************** checking database content... ok adding/updating host configuration... ok initializing the image database status... from the database from the front-end configuration from the database sql files updating the status of all databases initializing the repository skeleton list... ok initializing the signatures configurations list... ok Change current partition string '', enter a new partition string or 'cancel': new partition string to ''. (A)bort/(C)ontinue? [C]
The partitioning string should have the following structure:
Each token of that string, delimited by a ';' character, sets a repository to be split on one or several KIMA servers. The repository "REP_ID" will be split on KIMA servers with IDs KIMA_ID1 to KIMA_IDN.
REP_IDs are the real IDs of the repositories in the database. KIMA_ID is an integer between 1 and N, where N is the number of KIMA servers.
In order to partition repository 3 on KIMA servers 1, 2 and 3, the REPOSITORY_PARTITIONING string should look like:
Launching image database manager... ******************************************************************************* * Image Database Manager - Version 2.0 * * LTU Technologies - Copyright (c) 1999-2010 * * All rights reserved * ******************************************************************************* checking database content... no LTU schema creating the LTU schema... ok adding/updating host configuration... ok initializing the image database status... from the database from the front-end configuration from the database sql filesEd updating the status of all databases Network interface: eth0/eth0 addresses [ fe80:0:0:0:214:22ff:fe42:dd75%2/fe80:0:0:0:214:22ff:fe42:dd75%2/fe80:0:0:0:214:22ff:fe42:dd75%2 arcueil.ltutech.com/10.1.10.151/arcueil.ltutech.com ] Network interface: lo/lo addresses [ 0:0:0:0:0:0:0:1%1/0:0:0:0:0:0:0:1%1/0:0:0:0:0:0:0:1%1 localhost.localdomain/127.0.0.1/localhost.localdomain ] Local host address: arcueil.ltutech.com/10.1.10.151/arcueil.ltutech.com Change current partition string '', enter a new partition string or cancel:
Example: The repositories 10, 5, 4 and 1 are split upon 2 Kimas, Kima 1 and Kima 2:
Change current partition string '', enter a new partition string or cancel: 10:1,2;5:1,2;4:1,2;1:1;2;
Each host has to know what modules it is supposed to run. This is directly configured with "ltu-setup". You may also edit the configuration file "/etc/ltu/local.conf" of every host, and set the appropriate variables to 0 or 1 (run or do not run). Module variables are: WEKI_RUN, KIMA_RUN, FLXM_RUN, ADMI_RUN.
The distributed system installation is described in this chapter. You must repeat the following on each machine which is part of the distributed system. Before changing from a standalone installation to a distributed one, make sure that all of the LTU engine server Windows services are uninstalled.
Kima module access to the database with ODBC system.
Install PostGreSQL Odbc 64bits driver from c:\LTU\tools\install.
[ODBC] DRIVER=PostgreSQL Unicode(x64) UID=EDIT::Database_user_name PWD=EDIT::Database_user_password PORT=5432 SERVER=EDIT::Database_server_address DATABASE=EDIT::Database_name XaOpt=1 LowerCaseIdentifier=0 UseServerSidePrepare=0 ByteaAsLongVarBinary=0 BI=0 TrueIsMinus1=0 DisallowPremature=0 UpdatableCursors=1 LFConversion=1 ExtraSysTablePrefixes=dd_ CancelAsFreeStmt=0 Parse=0 BoolsAsChar=1 UnknownsAsLongVarchar=0 TextAsLongVarchar=1 UseDeclareFetch=0 Ksqo=1 Optimizer=1 CommLog=0 Debug=0 MaxLongVarcharSize=8190 MaxVarcharSize=255 UnknownSizes=0 Socket=4096 Fetch=100 ConnSettings= ShowSystemTables=0 RowVersioning=0 ShowOidColumn=0 FakeOidIndex=0 Protocol=7.4-1 ReadOnly=0 SSLmode=disable
[ODBC] DRIVER=SQL Server UID=EDIT::Database_user_name PWD=EDIT::Database_user_password AutoTranslate=No Network=DBMSSOCN Address=EDIT::Database_server_address,EDIT::port DATABASE=EDIT::Database_name APP=Microsoft Data Access Components SERVER=EDIT::Database_server_address
See Chapter 7, Configuring LTU engine server for details on all parameters that are used in this section
In the file c:\LTU\etc\ltu\global.bat:
Make sure the MAC address is correctly set (parameter LMHOSTID)
Set parameter DISTRIBUTED to 1
Set parameter DSN_FILE to c:\LTU\etc\ltu\database_postgres.dsn for PostGreSQL and c:\LTU\etc\ltu\database_mssql.dsn for MS SQL.
Depending on which modules you require for a specific machine, set the parameters FLXM_RUN, KIMA_RUN, WEKI_RUN and ADMI_RUN to 0 or 1.
If the machine is running a KIMA module, set its unique ID in the parameter KIMA_ID. IDs must have consecutive values between 1 and N where N is the number of KIMAs in your system.
If you use DNA 1,2,3,53 please change USEEXEOPTION to 1.
Run the tool db-manager.bat as described in Chapter 2.
You should manually start the system to verify that it is running correctly. To do this, use the command ltu-start_console.bat on each of the machines which are part of your system (as described in Chapter 4).
To run all modules as services you may install and run them as described in Chapter 5.
For security purpose, the system user "ltu" owns all LTU engine server files and processes. Still, you must be logged as "root" to start and stop the software.
LTU engine server is initially configured for a distributed installation. Additional configuration parameters are explained in Chapter 7, Configuring LTU engine server. They are marked as SKIPPED when ether you run the startup script "ltud".
Start LTU engine server
$ /etc/init.d/ltud start License server (23129) [ OK ] KImage Repository Module (23160) [ OK ] Web KImager Module (23179) [ OK ] ADMInistration Module (23500) [ OK ]
Test if the server is running
$ /etc/init.d/ltud status License server (23129) [ OK ] KImage Repository Module (23160) [ OK ] Web KImager Module (23179) [ OK ] ADMInistration Module (23500) [ OK ]
All activated modules must be running.
To restart LTU engine server, use this command : /etc/init.d/ltud restart
you can also start, get a status or restart only one module: /etc/init.d/ltud [start|status|restart] [weki|kima|admin]
To stop LTU engine server, you may use the same command, except that you use "stop".
$ /etc/init.d/ltud stop License Server (23129) [ OK ] KImage Repository Module (23160) [ OK ] Web KImager Module (23487) [ OK ] ADMInistration Module (23500) [ OK ]
you can also stop only one module: /etc/init.d/ltud stop [weki|kima|admin]
The first time you install the software, it is recommended to start the modules in console mode to check that every thing is running correctly. Once, your installation is validated, you may run the software from the Windows service interface.
To start and stop the services, you can use the "Service" program in the Windows Administration Tools. the names of services to launch are LTUADMI, LTUFLXM, LTUKIMA, and LTUWEKI.
You can also use the following scripts:
Open a command prompt window and change to directory "c:\LTU\bin" and type:
to start the services and
to stop the services.
The process monitoring is done through the MON module. This module checks that the activated modules are running and that the WEKI module answers some basic TCP and HTTP queries.
The MON process is handled by the startup script "/etc/init.d/mond", run it with the arguments start, status and stop to respectively start, check or stop the service.
To completely activate the process monitoring, you need to do the following:
Make sure MON is started
$ /etc/init.d/mond start Starting mon daemon: [ OK ]
Activate/Deactivate process monitoring using the administration interface
The MON administration interface is accessible through the administration interface (c.f. Section 4, “Web-based administration interface”).
The URL to access the interface is "
http://your_server_address:8005/" or "
https://your_server_address:8443/" when SSL is activated where "your_server_address" is Internet address of the host where LTU engine server is installed. The default administration ports may be changed in the configuration file /etc/ltu/local.conf.
Initially, LTU engine server is configured with the user "sadmin" (password "sadmin"), to properly setup your system you need to:
Log on the system with default user and password.
Change the "sadmin" rights and password (for security reasons).
Create content administration and search users.
The Figure 6.1, “Login page” shows the login page; simply type the login name and password ("sadmin" and "sadmin" by default).
The interface contains a menu bar on the top, by moving the mouse on the button "System Admin", you will see the system administration menu. This system administration interface includes the following pages:
Data Model Information
This interface let you manage the users and their rights. Figure 6.2, “User management page” shows the user management interface. By default, several users are included to the system. All of them having their password equals to their login name. The default users use default profiles that implement the different levels of possible user rights on the system.
To change a profile, just click on the button. Figure 6.3, “Profile management page” show the profile editing windows.
A profile contains two parts: global settings and per image database settings. The global rights part is shown in Figure 6.4, “Editing a profile: global settings”.
Figure 6.5, “Editing a profile: rights on the "Investigation" image database” shows the rights on the "Investigation" image database(repository).
This page is not actived by default in the system administration menu. To activate it, edit and modify the xml file "/etc/ltu/admi/databases/[your_repository_name]/[your_repository_name]_config.xml" on Linux or "[c:\LTU]\etc\ltu\admi\databases\[your_repository_name]\[your_repository_name]_config.xml" on Windows by setting the attribute "repositoryRole" to "disableProject".
All users' main actions are stored in the database; these pages allow the administrator to monitor the users' activity on LTU engine server. The two ways of viewing this activity are:
Reports. This tab gives a summary of user activity over a period of time.
Logs. This tab allows the analysis of each user action.
The figure above shows the user activity report, the 2 areas in this page are:
The search form. This search form is empty when you first go this page. To get all reports enter "%" in the user text entry and press the "Search" button.
The search result table. This table contains the user activity summary corresponding to the search criteria entered in the search form.
For each user activity report, you may press the button to get a detailed summary of this user activity.
The figure above shows the user activity logs, the 2 areas in this page are:
The search form. This search form is empty when you first go this page. To get all logs enter "%" in the user text entry and press the "Search" button.
The search result table. This table contains the user activity logs corresponding to the search criteria entered in the search form.
The export feature applies to all the logs and not only the current page or the selection. In the example above, the exported document will contain 81 logs.
This interface is based upon the "mon" open source software. If "mon" is not started, the web interface will give message saying it fails to connect to the monitoring process.
This intuitive interface enables you to see the activity of service and to trace back any failure.
Figure 6.9, “Process monitoring activation page” shows you the activation page.
This page displays the current main configuration parameters:
System configuration (database and license information): Figure 6.10, “System configuration: Global.conf file”
Current image database parameters: Figure 6.11, “System configuration: Ltu_global_conf table”
LTU engine server 5.0 stores now global parameters in a database table and comes with the possibility to configure the backend configuration from LTU engine server User Interface.
In the case of a distributed installation, backend configuration is centralized and relevant parameters are shared by all servers.
If you want to modify an existing parameter of
Ltu_gobal_conftable, change parameter values here:
Many administration programs are delivered with the product. Some of them are documented here.
The tool "db-manager" can be used to:
Install the schema in an empty database (db-manager is called during the installation process)
Install/uninstall one or more repositories
Activate/Desactivate one or more repositories
To run the database manager on Linux:
To run the database manager on Windows, run the script "[c:/LTU]/tools/install/db-manager.bat"
Several programs located in '
[LTU_HOME]/tools/perl/' may be of great help to manage the repository content and to create new scripts as well. To understand how to use each one, execute it with option '-h'.
The main programs:
Table 6.1. Perl tool programs
|Add images from a directory in a repository.|
|Empty an entity table from all its objects.|
|Add images and associated records.|
|Create an |
|Deletes an instance of an entity. For example, deletes the record with id 467.|
|Deletes an object set including external images.|
|Deletes the images that are not related to a record.|
|Downloads the key words from a repository and put them into a XML file.|
|To be used with Filter plugin: Filters the images of a directory.|
|Downloads the images of a repository and put them into a local directory.|
|Launches an internal task on an object set.|
|Submits the images of a directory to a repository and returns the image similarity score.|
|Updates the key words in the repository coming from a XML file.|
|searchImage.pl||Submits one image to a repository and returns the image similarity score.|
|statBounceImage.pl||Bounce on N images from the database displaying stats.|
|compareImages.pl||Compare 2 images and return score with associated fine comparison analysis files.|
|statSearchImage.pl||Search all directory images displaying stats.|
On top of the standard Perl installation packages, optional packages must be installed, see Table 4.3, “Required RPMs on Linux systems”.
In case errors occur in LTU engine server, you have the possibility to generate automatically an incident report archive. This archive can be send to LTU support at
email@example.com and will speed up the diagnose step and resolution of your problem.
To do that:
$ cd LTU_HOME/tools/incident/ $ ./incidentArchive.sh Building the incident directory Creating the Tar file Incident Archive finished ***** ***** You can find the incident archive in ***** Incident_log_YYYY_MMDD_HH_MM_SS.tgz ***** Please send it to the LTU support *****
You can also use the diagnostic tool accessible from LTU engine server User Interface for generating the incident report archive.
System > Diagnostic Tool gives you access to a page on which you can select which log and configuration files you want to download from LTU engine server for diagnostic and troubleshooting purposes. Click on
Diagnostic downloads one archive file with the requested files.
LTU engine server includes a task management module that enables to run administration tasks within the backend.
The task service configuration file is "
[c:/LTU]/etc/ltu/weki-scheduler.xml". This file declares how and when tasks must be ran.
<?xml version="1.0" encoding="UTF-8"?> <task-scheduler xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="xml-backingstore.xsd"> <!-- Remove this comments to activate AutoSync and be sure to edit the configuration file <task name="Auto Synchronization"> <class>com.ltu.taskservice.impl.FileSystemSynchro</class> <style>cron</style> <cron-expression>0 55 01 * * ?</cron-expression> <parameters> <key type="string" name="autosync.file.settings" value="/etc/ltu/autosync.properties"/> </parameters> </task> --> <task name="Log maintenance"> <class>com.ltu.components.taskservice.impl.LogMaintenance</class> <style>cron</style> <cron-expression>0 55 01 * * ?</cron-expression> <parameters> <key type="string" name="log.life_expectancy" value="365"/> </parameters> </task> </task-scheduler>
By default, two tasks are included in LTU engine server 5.0:
The AutoSync task that will synchronize an image database with the content of a directory. Every new image added in the directory (or in a subdirectory) will be added in the image database. Every image remove from the directory will be removed from the image database. This task does not handle image meta info and is intended to clients who are using only LTU engine server backend. This task is disabled by default. The configuration file "
/etc/ltu/autosync.properties" contains the parameter for this task.
The log maintenance task that is executed every night to clean the database for user log entries older than a certain amount of days (1 year by default).
For those unfamiliar with "cron", this means being able to create a firing schedule such as: "At 8:00am every Monday through Friday" or "At 1:30am every last Friday of the month". A "Cron-Expression" is a string comprised of 6 or 7 fields separated by white space. The 6 mandatory and 1 optional fields are as follows:
Table 6.2. Cron-expression parameters
|Field Name||Allowed Values||Allowed Special Characters|
|Seconds||0-59||, - * /|
|Minutes||0-59||, - * /|
|Hours||0-23||, - * /|
|Day-of-month||1-31||, - * / ? L W C|
|Month||1-12 or JAN-DEC||, - * /|
|Day-of-week||1-7 or SUN-SAT||, - * / ? L C #|
|Year (Optional)||empty, 1970-2099||, - * /|
The '*' character is used to specify all values. For example, "*" in the minute field means "every minute".
The '?' character is allowed for the day-of-month and day-of-week fields. It is used to specify 'no specific value'. This is useful when you need to specify something in one of the two fields, but not the other. See the examples below for clarification.
The '-' character is used to specify ranges For example "10-12" in the hour field means "the hours 10, 11 and 12".
The ',' character is used to specify additional values. For example "MON,WED,FRI" in the day-of-week field means "the days Monday, Wednesday, and Friday".
The '/' character is used to specify increments. For example "0/15" in the seconds field means "the seconds 0, 15, 30, and 45". And "5/15" in the seconds field means "the seconds 5, 20, 35, and 50". You can also specify '/' after the '*' character - in this case '*' is equivalent to having '0' before the '/'.
The 'L' character is allowed for the day-of-month and day-of-week fields. This character is short-hand for "last", but it has different meaning in each of the two fields. For example, the value "L" in the day-of-month field means "the last day of the month" - day 31 for January, day 28 for February on non-leap years. If used in the day-of-week field by itself, it simply means "7" or "SAT". But if used in the day-of-week field after another value, it means "the last xxx day of the month" - for example "6L" means "the last Friday of the month". When using the 'L' option, it is important not to specify lists, or ranges of values, as you'll get confusing results.
The 'W' character is allowed for the day-of-month field. This character is used to specify the weekday (Monday-Friday) nearest the given day. As an example, if you were to specify "15W" as the value for the day-of-month field, the meaning is: "the nearest weekday to the 15th of the month". So if the 15th is a Saturday, the trigger will fire on Friday the 14th. If the 15th is a Sunday, the trigger will fire on Monday the 16th. If the 15th is a Tuesday, then it will fire on Tuesday the 15th. However if you specify "1W" as the value for day-of-month, and the 1st is a Saturday, the trigger will fire on Monday the 3rd, as it will not 'jump' over the boundary of a month's days. The 'W' character can only be specified when the day-of-month is a single day, not a range or list of days.
The 'L' and 'W' characters can also be combined for the day-of-month expression to yield 'LW', which translates to "last weekday of the month".
The '#' character is allowed for the day-of-week field. This character is used to specify "the nth" XXX day of the month. For example, the value of "6#3" in the day-of-week field means the third Friday of the month (day 6 = Friday and "#3" = the 3rd one in the month). Other examples: "2#1" = the first Monday of the month and "4#5" = the fifth Wednesday of the month. Note that if you specify "#5" and there is not 5 of the given day-of-week in the month, then no firing will occur that month.
The 'C' character is allowed for the day-of-month and day-of-week fields. This character is short-hand for "calendar". This means values are calculated against the associated calendar, if any. If no calendar is associated, then it is equivalent to having an all-inclusive calendar. A value of "5C" in the day-of-month field means "the first day included by the calendar on or after the 5th". A value of "1C" in the day-of-week field means "the first day included by the calendar on or after Sunday".
The legal characters and the names of months and days of the week are not case sensitive.
Here are some full examples:
Table 6.3. Cron-expression examples
|"0 0 12 * * ?"||Fire at 12pm (noon) every day|
|"0 15 10 ? * *"||Fire at 10:15am every day|
|"0 15 10 * * ?"||Fire at 10:15am every day|
|"0 15 10 * * ? *"||Fire at 10:15am every day|
|"0 15 10 * * ? 2005"||Fire at 10:15am every day during the year 2005|
|"0 * 14 * * ?"||Fire every minute starting at 2pm and ending at 2:59pm, every day|
|"0 0/5 14 * * ?"||Fire every 5 minutes starting at 2pm and ending at 2:55pm, every day|
|"0 0/5 14,18 * * ?"||Fire every 5 minutes starting at 2pm and ending at 2:55pm, AND fire every 5 minutes starting at 6pm and ending at 6:55pm, every day|
|"0 0-5 14 * * ?"||Fire every minute starting at 2pm and ending at 2:05pm, every day|
|"0 10,44 14 ? 3 WED"||Fire at 2:10pm and at 2:44pm every Wednesday in the month of March.|
|"0 15 10 ? * MON-FRI"||Fire at 10:15am every Monday, Tuesday, Wednesday, Thursday and Friday|
|"0 15 10 15 * ?"||Fire at 10:15am on the 15th day of every month|
|"0 15 10 L * ?"||Fire at 10:15am on the last day of every month|
|"0 15 10 ? * 6L"||Fire at 10:15am on the last Friday of every month|
|"0 15 10 ? * 6L 2002-2005"||Fire at 10:15am on every last Friday of every month during the years 2002, 2003, 2004 and 2005|
|"0 15 10 ? * 6#3"||Fire at 10:15am on the third Friday of every month|
Pay attention to the effects of '?' and '*' in the day-of-week and day-of-month fields!
Support for the features described for the 'C' character is not complete.
Support for specifying both a day-of-week and a day-of-month value is not complete (you'll need to use the '?' character in on of these fields).
Be careful when setting fire times between mid-night and 1:00 AM - "daylight savings" can cause a skip or a repeat depending on whether the time moves back or jumps forward.
The configuration of LTU engine server is done through configuration files located in "[c:/LTU]/etc/ltu" and through the database table
In this chapter all configuration options are detailed.
This section lists standard configuration parameters, as well as advanced configuration parameters that should not be edited unless under the supervision of an LTU support and consulting team (firstname.lastname@example.org).
All variables are listed as in table below
Table 7.1. Parameters notation
|Variable name||default value||value type or possible values||Some remarks on the use of this parameter|
The configuration files are shell files. Two rules must be followed:
All comment must be on a line starting with "#".
Variable must be assigned with no spaces between the variable name, the sign "=" and the value. For example, "SERVER_NAME=my_server" is good and "SERVER_NAME= my_server" will produce an error.
The configuration files are:
Table 7.2. Main configuration files
|install.conf (Linux only)||Local installation directory (not to be edited)|
|global.conf (Linux only) or global.bat (windows only)||Configuration file common to all hosts in the LTU engine server system (this file should be the same on each host).|
|local.conf (Linux only)||Local configuration|
|database.conf (Linux only)||The database connection properties (not to be edited)|
The database table which stores global configuration variables is
Table 7.3. Install.conf parameters
By default, images (thumbnails and originals) and binary are stored in "[c:/LTU]/var/ltu/repXXX/" (default repository settings), signatures are stored in database.
The content administrator may choose to store them directly in the database (PostGreSQL or MSSQL).
Table 7.4. Global.conf or global.bat installation parameters
|HTTP_ADDR (Linux only)||None||None server name or IP address||Put the WEKI server address for standard use.|
|DISTRIBUTED||yes on Linux, 1 on Windows||yes or no on Linux 0 or 1 on Windows||Tells the system to work distributed or not|
|KIMA_64 (Windows only)||1||0 or 1||1 to run 64bits KIMA server (only on 64bits host), 0 to run 32bits KIMA server (on 32bits or 64bits host)|
Table 7.5. Global.conf or global.bat database parameters
|DATABASE||POSTGRESQL||POSTGRESQL or MSSQL(Windows only)||Database used|
|DATABASE_ADDR||None||Server name or IP address||Host name or IP address of the database server|
|DATABASE_NAME||None||Database identifier||Name of the database|
|DATABASE_PORT||5432 (PostGreSql default port)||Port number||Database port (5432 is the default PostGreSql one, 1433 is the default MSSQL port )|
|DATABASE_LOGINNAME||None||User login||Database user login|
|DATABASE_LOGINPWD||None||User password||Database user password|
The configuration file "local.conf" contains the parameters specific to each host of the LTU engine server installation
These parameters exist and are also configured with the same values in global.bat (Windows).
The variable "*_RUN" are used to activate or not each module on each host. When modules are activated they are started by the "ltud" command, otherwise the module is ignored and the status "skipped" is always displayed.
Table 7.6. local.conf installation parameters
|WEKI_RUN||1||0 or 1||WEKI module activation|
|KIMA_RUN||1||0 or 1||KIMA module activation|
|FLXM_RUN||1||0 or 1||License server module activation|
|ADMI_RUN||1||0 or 1||Administration module activation|
|MON_RUN (Linux only)||1||0 or 1||Enable process monitoring|
The variable KIMA_ID identifies each KIMA module and must be unique among all KIMA modules of the system. Each module loads the categories according to this id. Two KIMA modules having the same id would load the same images in memory leading to conflicts.
Table 7.7. local.conf KIMA parameters
|KIMA_PORT||8091||Number||KIMA listening port|
|KIMA_ID||1||Number||The KIMA unique identification number. Must be <= KIMA_NUMBER|
|KIMA_NUM_THREADS||10||Number||Number of thread for kima. Must be equal to 2 times the number of core allocated to the kima(0: unlimited)|
Table 7.8. local.conf WEKI parameters
|WEKI_MEM_SIZE||1500 on Linux 512 on Windows||Number||Maximum memory size allowed for the java virtual machine in MBytes.|
|WEKI_PORT||8090||Port number||Listening port for LTU engine server request|
|WEKI_DATABASE_POOLSIZE (Linux) DATABASE_POOLSIZE (Windows)||5 on Linux 4 on Windows||Number||Database connection pool size (do not edit).|
|WEKI_NUM_THREADS||10||Number||Number of threads allocated to this weki (i.e. number of parallel processing). It must be consistent with your license and with the configuration of the other ltu modules (the kima server(s) and/or the other weki server(s)).|
Table 7.9. local.conf ADMI parameters
|USE_SYSTEM_HTTPD (Linux only)||1||Number|
|USE_SSL (Linux only)||0||0 or 1||Secure web based administration with SSL|
|ADMI_APACHE_PORT (Linux only)||8005||Port number||Administration listening port|
|ADMI_APACHE_SSL_PORT (Linux only)||8443||Port number||Administration listening port when using SSL.|
The process monitor program "mon" is by default only watching processes to determine whether they are running or not. Additionally, it can also monitor connections to the WEKI module to verify regularly if the HTTP module is up and ready.
Table 7.10. Install.conf MON parameters
|MON_WEKI_SERVICE_TCP (Linux only)||1||0 or 1||Monitor the WEKI module attempting TCP connection regularly.|
|MON_WEKI_SERVICE_HTTP (Linux only)||0||0 or 1||Monitor the WEKI module attempting HTTP connection regularly.|
|ADMINISTRATOR_MAIL (Linux only)|
The parameters in
ltu_global_conf table are advanced configuration parameters. After changing any of these values, you must restart the backend. Please for more informations, contact LTU support LTU at
Table 7.12. Variables handled by ltu_global_conf table
|api.version||4.0||Do not edit|
|classification.classif.threshold||Keyword suggestion classification threshold|
|classification.enabled.repositories||Repositories on which keyword suggestion is activated|
|db.version||4.7||Do not edit|
|http.max.post.size||Maximum image size in byte|
|http.response.encoding||Backend response encoding|
|image.encryption.enable||Encrypt images in database - yes or no|
|image.max.surface||Maximum image surface in pixels|
|image.min.size||Maximum image height or width|
|keyword.metric.coefficient||Do not edit|
|keywordpropagation.enabled.repositories||Repositories on which keyword suggestion is activated|
|repository.split||Distribution partitioning string|
You can also view the variables handled in ltu_global_conf table from LTU engine server User Interface from the menu
System > Configuration > System.
For a repository NN, an entry in the table repNN_ltu_rep_conf contains the repository parameters which are detailed in Table 7.13, “Repository configuration parameters”.
Table 7.13. Repository configuration parameters
|signature_home_dir, signature_storage, image_home_dir, image_storage, binary_storage,||-||-||See Table 7.14, “Storage location database attributes”.|
|binary_home_dir||/var/ltu/repNN/binary/||directory||Where external images are stored. These images can only be stored on file system in this version. They are images from AutoScan search reports.|
|default_segment_image||0||0 or 1||Whether to segment images or not (0 means do not segment)|
|default_color_weight||50||Integer between 0 and 100||Default color weight used when not specified in the query (bounceimage or searchimage)|
|default_prefilter_image||0||0 or 1||Whether to prefilter images or not (0 means do not prefilter)|
|default_nb_results||20||Integer||Default number of results when not specified in the query.|
|nb_results_cached||100||Integer||Number of results cached.|
|default_signature_type||-||Valid signature number||Refer to Chapter 2, LTU engine server Overview for more information about signature ids.|
|quantize||-||-||Not used anymore.|
|use_stemming||0||0 or 1||Whether or not stem keywords (e.g. girl equals to girls)|
|use_image_meta_infos||0||0 or 1||Whether or not extract EXIF and IPTC information from images.|
|store_original_image||1||0 or 1||Whether or not keep original images|
|store_thumbnail_image||1||0 or 1||Whether or not compute and store thumbnail images.|
|retrieval_threshold||-1.0||-1.0 or a float > 0||Threshold on retrieval results. Image with a score superior to this threshold will NOT be returned. -1.0 means no threshold.|
|similar_threshold||1.2||Float > 0||AutoScan similar threshold|
|clone_threshold||1.0||Float > 0||AutoScan clone threshold|
|video_match_threshold||1.1||Float > 0||Video matching threshold.|
|nb_retrieval_threads||1||Integer||Number of threads used in the retrieval service|
|use_combined_keyword_distance||0 (1 for signature 2)||Integer (0 or 1)||use combined keyword distance|
Any change in this table will be take into account at the next service restart.
LTU engine server 5.0 can store images, signatures and binary on disk or in the database (available with POSTGRESQL and MSSQL). The storage location is configured in the database. Configuration must be done when the image database is empty.
For example, the default image database "photo gallery" is configured to store images and signatures on disk in the directory "[c:/LTU]
For each repository defined in the database, a table named "REPNN_LTU_REP_CONF" contains the main configuration parameters for the repository. In this table, the storage configuration is done with the following attributes defined in Table 7.14, “Storage location database attributes”, the examples come from the default image database "photo bank" which repository id is NN.
Table 7.14. Storage location database attributes
|SIGNATURE_HOME_DIR||[c:/LTU]/var/ltu/rep1/signatures/||directory||Used only if SIGNATURE_STORAGE is set to "file".|
|SIGNATURE_STORAGE||file||file or db||If set to "db", the signatures are stored in the database.|
|IMAGE_HOME_DIR||[c:/LTU]/var/ltu/rep1/images/||directory||Used only if IMAGE_STORAGE is set to "file".|
|IMAGE_STORAGE||file||file or db||If set to "db", the images are stored in the database.|
|BINARY_HOME_DIR||[c:/LTU]/var/ltu/rep1/images/||directory||Used only if BINARY_STORAGE is set to "file".|
|BINARY_STORAGE||file||file or db||If set to "db", the binary are stored in the database|
All the interface configuration files are located in the directory "
The main properties that may be configured from a system administrator point of view are in
<!-- Number of queries displayed in the user search history. This is not linked with the real history size, defined in the backend by the LogsMaintenance. --> <PROPERTY name="search_history_size" type="int" value="20"/> [...] <!-- Number of objects displayed in a print project window --> <PROPERTY name="print_project_display_number" type="int" value="500"/> [...] <!-- For mass-edition, mass-deletion or mass-copy, limit the selection size to avoid time limit in php. --> <PROPERTY name="default_selection_limit" type="int" value="2000"/> [...] <!-- Number of seconds of inactivity before the session expire --> <PROPERTY name="session_expiration_time" type="int" value="30"/>
It is possible to secure the access to the web-based interface using SSL. LTU engine server is delivered with a default SSL certificate that should be changed for maximum security. In particular, when you access the LTU engine server server through, your browser should display a warning message (see Figure 7.1, “Internet Explorer warning.”) saying that:
The certificate issuer is unknown to your browser.
The server name is not the registered name in the certificate.
The user can ignore this warning and continue. However, you may decide to handle SSL certificates yourself.
LTU engine server external user tools "Image Enrollment", "Image-Cropper" and "AutoScan" supports SSL with a server certificate only. It is not possible currently to configure SSL to require client certificates with these tools.
In this section, we explain how to change LTU engine server WEKI and ADMI services' listening ports.
You only need to change the appropriate parameter in
ADMI_APACHE_SSL_PORT if running the interface in secure mode. Then you need to restart the service ADMI.
First change the WEKI listening port in
local.conf (Linux only): parameter
WEKI_PORT. Then you need to change references to the WEKI service in the admi configuration:
[c:/LTU]/etc/ltu/admi/databases/rep_name/repXXX_config.xml, set the "
host" property of the SERVER element to the address of the WEKI server :
<SERVER host "" port="8090" apiVersion="4.0" logLevel="-1" retry="0" timeout="0"/>
In the directory
[c:/LTU]/etc/ltu/admi/httpd/, change the apache configuration file
httpd_1.3.conf (Red Hat ES 2.1 only),
httpd_2.2.conf, change the proxy setting of Apache to redirect all LTU engine server API queries on the WEKI :
ProxyPass /iseeker/ http://localhost:8090/iseeker/ ProxyPass /ifilter/ http://localhost:8090/ifilter/ ProxyPass /common/ http://localhost:8090/common/
In this chapter, we describe how the software runs on the system: the files and directories, the processes, the port opened...
The notation "[HOME_LTU]" must be replaced by your installation directory.
On Linux, the installation process adds the user "ltu" to the system. This user owns all LTU engine server files and directories. The system user "ltu" also owns all LTU engine server processes.
The program to start, stop and status the LTU engine server services is "/etc/init.d/ltud".
Usage: /etc/init.d/ltud [start|stop|restart|status]
The system uses this program to start LTU engine server when the system starts up and to stop it when the system shut down using following symbolic links:
/etc/rc3.d/S98ltud -> /etc/init.d/ltud
/etc/rc4.d/S98ltud -> /etc/init.d/ltud
/etc/rc5.d/S98ltud -> /etc/init.d/ltud
/etc/rc0.d/K10ltud -> /etc/init.d/ltud
/etc/rc1.d/K10ltud -> /etc/init.d/ltud
/etc/rc2.d/K10ltud -> /etc/init.d/ltud
/etc/rc6.d/K10ltud -> /etc/init.d/ltud
The program "/etc/init.d/ltud" runs the programs described below under the user "ltu" (these scripts can be called directly). To understand how to use each one, execute it with option '-h'.
Log files are located in the directory "
Table 8.1. Log file by module
|ADMI||admi.log, admi_error.log, admi_access.log, admi_user_access.log|
|WEKI||weki.log, weki0.log, weki1.log...|
At each software startup, these log files are saved in the same directory. The date is used as the extension in backup log files name, e.g. "kima.log.20030518-20:17:22".
This section describes the content and role of the LTU engine server directories and files.
Directories with comments:
bin: This directory contains all startup files.
Contribs: This directory contains all external packages.
Contribs/jakarta-tomcat Tomcat used by the "weki" module.
Contribs/jakarta-tomcat/server/lib: contains LTU engine server java libraries.
Contribs/perl-libs, Contribs/Smarty, Contribs/j2sdk1.5, Contribs/Torque..., Various external libraries used by modules.
Contribs/mon-0.99.2 The process monitoring software(Linux only).
Contribs/apache-ant Ant tools for external plugin use.
Contribs/apache_php The apache server used for the "admi" module (the web-based interface).
tools: This directory contains various software tools to administrate LTU engine server. In particular all the database scripts used for the install and uninstall are located in the subdirectory "database". This directory also includes db-manger tool located in subdirectory "install". This tool is used to manage the repositories.
ltuSystems: This directory contains LTU engine server binaries and libraries.
ltuSystems/native/ External and internal native libraries.
ltuSystems/http_data/htdocs/ Web-based interface files and libraries.
Each LTU engine server module runs one or several processes.
On Linux, the user "ltu", except for the process-monitoring service "mon" described in a later section, owns all processes.
All the processes use libraries and data located in "[HOME_LTU]" and may access configuration files in "[c:/LTU]/etc/ltu".
This section describes the system when configured to use the ORB JacORB (default configuration).
Table 8.2. Running program and port opened by module
|flxm||[HOME_LTU]/bin/win32/lmgrd, [HOME_LTU]/bin/win32/ltutech||27000 + Arbitrary port||These programs access the license file in "/etc/ltu".|
|kima||[HOME_LTU]/bin/win32/StandalonekimaServer or [HOME_LTU]/bin/win64/StandalonekimaServer||8091|
|weki||[HOME_LTU]/Contribs/ j2sdk1.5/bin/java||8090 + Arbitrary port|
|admi||[HOME_LTU]/Contribs/ apache_php/bin/httpd or /usr/sbin/httpd||8005 or 8443 with SSL||Use first program on Red-Hat Enterprise Server 2.1, use the second otherwise|
The modules KIMA and WEKI use SOAP to communicate with each other: several ports may be opened at any time by any of these services.
The process monitoring service is run by the system at startup via the script "/etc/init.d/mond". This script dynamically write its own configuration file (based on the local LTU engine server settings) in "[HOME_LTU]/Contribs/mon". The «root» user always owns this service.
This service uses several utility scripts to check the status of LTU engine server services and eventually restart modules if they are not running or if they fail answering basic queries. The table below describes the main characteristics of these programs.
Programs: /usr/bin/perl, /bin/sh
Write access: [HOME_LTU]/Contribs/mon
Read access: [HOME_LTU]/Contribs/perl-libs [HOME_LTU]/run
Execution access: [HOME_LTU]/bin
Network access: IP 127.0.0.1 Port 8090
The uninstall program "ltu-uninstall" is provided with the distribution on the CDROM or in the archive file. Follow the pre-installation tasks of the installation procedure to get access to this program.
Before uninstalling the LTU engine server, make sure that you use the uninstall program provided with the version of the LTU engine server you want to uninstall. (Especially when you need to uninstall LTU engine server for an upgrade, use the uninstall program of the LTU engine server you are about to upgrade).
First, change to the install directory
$ cd /tmp/LTUEngineServerInstall
$ cd /mnt/cdrom
Then, run ltu-uninstall
$ ./ltu-uninstall ***************************************************************************** * * * LTU-Engine Server uninstall * * * * Copyright (c) 1999-2011 LTU Technologies * * All rights reserved * * * ***************************************************************************** -------------------------------------------------------------------- 1: Shell libraries -------------------------------------------------------------------- Loading LTU Technologies main shell library [ OK ] Loading Linux shell library [ OK ] Loading database shell library [ OK ] Loading LTU Technologies install shell library [ OK ] Uninstall log file: /tmp/ltu-uninstall.260308.log Refer to this file if you encounter any problem during the uninstall process. -------------------------------------------------------------------- 2: Launch uninstall -------------------------------------------------------------------- Looking for an installed LTU-Engine Server version [ OK ] -------------------------------------------------------- Uninstall <LTU-Engine-Server> (A)bort/(C)ontinue? [C] C
The uninstall program indicates the LTU engine server version currently installed on the host and ask you if you really want to uninstall LTU engine server.
Do you want to backup your configuration (Y)es/(N)o? [Y] Y Enter a backup directory [/tmp/backup]: /tmp/MyBackUp Creating directory /tmp/backup [ OK ] Backup main configuration directory /etc/ltu [ OK ] Configuration backup of LTU-Engine Server [ OK ]
At this point you may choose to backup you configuration files (All files in "/etc/ltu"). If you want to do so, "ltu-uninstall" will ask you to enter the backup directory.
Stopping mon [ OK ] Stopping all modules [ OK ] -- Default database storage with the following parameters: - Database user login : ltu_user - Database user password : ltu_password Checking postgresql client binaries [ OK ] Do you want to uninstall the database (Y)es/(N)o? [N] Y Uninstalling the database will erase all your data (A)bort/(C)ontinue? [C] C
The program indicates the current database parameters and asks you whether you want to uninstall the database. If you choose to uninstall the database, the program asks to confirm.
If you choose to uninstall the database, all tables and their content will be deleted included all information related to signatures and images you were using with LTU engine server.
Uninstalling the database will erase all your data (A)bort/(C)ontinue? [C] Checking needed packages for database module [ OK ] Loading database storage [ OK ] Restarting database storage [ OK ] Checking if user ltu_user exists [ OK ] Drop database ltu_engine [ OK ] Remove image and signature directory /var/ltu [ OK ] Status of LTU engine server database uninstall: [ OK ] -------------------------------------------------------------------- 2.1: LTU-Engine Server Software uninstall -------------------------------------------------------------------- Remove software files (/etc/ltu,/home/ltu,/var/log/ltu,/var/run/ltu) (A)bort/(C)ontinue? [C] C Removing directories - /etc/ltu [ OK ] - /home/ltu [ OK ] - /var/log/ltu [ OK ] - /var/run/ltu [ OK ] -------------------------------------------------------------------- 2.2: Clean-up the machine -------------------------------------------------------------------- Delete user ltu (A)bort/(C)ontinue? [C] C Delete user ltu [ OK ] Unregister LTU-Engine-Server services [ OK ] Unregister mon service [ OK ] Delete ltu service system start script [ OK ] Delete mon service system start script [ OK ] -------------------------------------------------------------------- Status of LTU-Engine Server Software uninstall: [ OK ] -------------------------------------------------------------------- STATUS: HOST your_hostname UNINSTALL ENDED SUCCESSFULLY
The uninstall program stop all processes and remove all LTU files. You may choose to skip any of the uninstall step by answering "A" to the confirmation questions.
Before uninstalling the software you should make sure that the LTU services are not running anymore.
Then, open a command prompt window and change to directory c:\LTU\tools\install. Run the following script:
This script should uninstall the four Windows services named "LTUADMI (Apache 2)", "LTUFLXM", "LTUKIMA" and "LTUWEKI (Tomcat server 5)".
To completely remove LTU engine server from the machine, you can simply remove the directory "c:/LTU". If you want to keep your data, you should before save "c:/LTU/var/ltu".
These are the image formats supported by LTU engine server. For most of these formats, LTU engine server does not use the extension of the file to recognize the format; it uses special information inside the binary content (i.e. a JPEG image rename with an ".exe" extension will be recognized as an image). For some image formats, LTU engine server needs the file extension; these formats are marked with "*".
Table A.1. Supported Image Format
|BMP||Microsoft Windows bitmap||bmp, dib, vga, bga, rle, rl4, rl8|
|DCX||Zsoft IBM Multi-page Paintbrush||dcx|
|FITS||Flexible Image Transport System (not supported in thi release)||fits|
|GIF||CompuServe Graphics Interchange||gif, giff|
|JP2||JPEG-2000 JP2 (linux only)||jp2|
|JPEG||Joint Photographic Experts Group||jpeg, jpg, jfif|
|MIFF||Magick image file format||miff|
|PCX||ZSoft IBM PC Paintbrush||pcx, pcc|
|PICT||Apple Macintosh QuickDraw||pict, pict2|
|PIX*||Alias/Wavefront RLE image format||pix|
|PNG||Portable Network Graphics||png|
|PSD||Adobe Photoshop bitmap file||psd|
|SUN||Sun Rasterfile||ras, rast, sun, sr, scr|
|SCT*||Scitex Continuous Tone||sct|
|RLE||Utah Run Length encoding||rle|
|TGA*||Truevision Targa||tga, vda, icb, targa, vst|
|TIFF||Tagged-Image File Format||tif, tiff|
|VIFF||Khoros Visualization||viff, xv|