LTU engine Server V 5.0 - Installation / Administration Guide

LTU Technologies

Legal Notice

LTU engine Server™ is Copyright © 1999-2011 by LTU Technologies

Revision History
Revision $Revision: 1.113 $

Abstract

This guide is intended to system administrators and applies to LTU engine server 5.0


1. Introduction
1. What's New in LTU engine server 5.0
2. LTU Technologies Contacts
3. Reference Documents
4. Glossary
5. Acronyms
6. Known Restrictions in this Build
7. Third Party Copyrights
2. LTU engine server Overview
1. Technology Overview
2. LTU engine server Features
3. Software Architecture
3.1. Overview: a distributable components architecture
3.2. The KIMA module
3.2.1. Standard distribution
3.2.2. Distribution with partitioning of repository(ies)
3.3. The WEKI module
4. LTU engine server signatures
3. Setting Up the Database
1. Microsoft SQL Server 2005
2. PostGreSql
4. Installing or upgrading LTU engine server
1. System Requirements
2. Supported configurations
3. Software Requirements on Linux
4. Distributed or standalone installation
5. Upgrading from LTU engine server previous versions
5.1. Database schema upgrade
5.2. Frontend configuration file upgrade
6. Installing on Linux
6.1. Pre-installation tasks
6.2. Installing and configuring LTU engine server
6.3. Configure syslog
7. Installing on Windows
7.1. Pre-installation tasks
7.2. Installing and configuring LTU engine server
8. LTU engine server license
8.1. Licensing mechanism
8.2. Getting information for an LTU engine server system
8.2.1. Getting the MAC address on Linux
8.2.2. Getting the number of CPU(s) of a machine
8.2.3. Getting the host name of a machine on Linux
8.3. The FLXM module (license(s) server)
8.4. Requesting a license
8.4.1. Mono host installation (standard installation)
8.4.2. Multi hosts installation (custom installation)
8.5. Installing the license file
8.6. Log messages on license check
9. Maintenance
9.1. How to backup LTU engine server
9.2. How to restore LTU engine server
9.3. How to use a new LTU engine server version with previous database
5. Deploying a distributed LTU engine server system
1. Architecture design
1.1. ADMI module
1.2. WEKI module
1.3. KIMA module
1.4. Designing the architecture
2. Deploying and installing all the hosts on Linux
2.1. Configure ODBC access to the database
2.1.1. PostGreSql
2.2. KIMA configuration
2.3. ADMI configuration
2.3.1. Distribution of ADMI modules
2.3.2. Tested Configuration
2.4. Module activation and repository partitioning settings
3. Deploying and installing all the hosts on Windows
3.1. Configure ODBC access to the database
3.1.1. PostGreSQL
3.1.2. MSSQL
3.2. Configuring the System
3.3. Start and Test the System
4. Initialization of the system
6. Administrating LTU engine server
1. Starting/stopping LTU engine server on Linux
1.1. Starting LTU engine server
1.2. Stopping LTU engine server
2. Starting/stopping LTU engine server on Windows
3. Activating/Deactivating process monitoring (Linux only)
4. Web-based administration interface
4.1. Rights Management
4.2. User Activity
4.3. Monitoring LTU engine server processes (Linux only)
4.4. Configuration parameters
5. Administration tools
5.1. Database manager
5.2. Perl programs for managing repository content
5.3. Incident report archive (Linux only)
6. The task service
7. Configuring LTU engine server
1. System configuration files
1.1. Install.conf (Linux only)
1.2. Global.conf or global.bat
1.2.1. Global installation parameters
1.2.2. Database parameters
1.3. Local.conf (Linux only)
1.3.1. Local installation parameters
1.3.2. KIMA local parameters
1.3.3. WEKI local parameters
1.3.4. ADMI local parameters
1.3.5. MON local parameters
1.3.6. Other parameters
1.4. Database.conf (Linux only)
1.5. Global parameters in "ltu_global_conf" table
2. Repository configuration "repNN_ltu_rep_conf"
3. Setting the storage location
4. Graphical user interface configuration
5. Using SSL (Linux only)
6. Changing service listening ports on a standalone application
6.1. ADMI listening port (Linux only)
6.2. WEKI listening port
8. LTU engine server files and processes
1. Software controls (Linux only)
2. Software log files
3. Files and directories
3.1. Directory [HOME_LTU]
3.2. Directory "[c:/LTU]/etc/ltu"
3.3. Image and signature directory
4. Processes
5. The process-monitoring service "mon" (Linux only)
9. Uninstalling LTU engine server
1. Uninstalling on Linux
2. Uninstalling on Windows
A. Supported Image Formats

Chapter 1. Introduction

This guide is intended to system administrators and applies to LTU engine server 5.0.

1. What's New in LTU engine server 5.0

The release notes for LTU engine server 5.0 can be downloaded from the LTU Technologies download center.

2. LTU Technologies Contacts

Please contact support@LTUtech.com for any questions.

3. Reference Documents

LTU-ES-5.0-AGLTU engine server 5.0 Administration GuideThis document

4. Glossary

SignatureBinary encoding of visual features (texture, color, shape, etc.).
Image DNASee signature. Please note that "image DNA" and "image signature" are synonyms. "Image DNA" is used in general documentation, while "Image Signature" is used in technical documentation.
KimageThe binary object attached to each image in the system. It contains a unique identifier, the signature, the MD5 and the reference to access the original and thumbnail images. It may also contains the keywords and the IPTC or EXIF data if any.
RepositoryEvery kimage in the system is assigned to a repository. A visual search is done in one, and only one, repository.

5. Acronyms

CBIRContent-based image retrieval
GUIGraphical User Interface
SOAPSimple Object Access Protocol

6. Known Restrictions in this Build

  • AutoScan client is only available for Windows users.

    Please contact LTU Professional Services to obtain the Linux package.

7. Third Party Copyrights

PostgreSQL is Copyright © 1996-2008 by the PostgreSQL Global Development Group and is distributed under the terms of the license of the University of California

Java, J2RE, J2SE, and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Red Hat is a registered trademark of Red Hat, Inc.

Chapter 2. LTU engine server Overview

Digital content has become a vital enterprise asset, and visual content management has emerged as a strategic necessity. LTU provides the advanced technologies and solutions that address this new imperative, enabling companies to manage and use visual assets more effectively.

Based upon LTU's innovative image recognition technology, LTU engine server simplifies the search process by enabling more intuitive navigation. It allows users to perform visual searches by images (instead of by name, keyword, number or description).

LTU engine server provides businesses with a competitive advantage by both cutting internal costs for database search and creating new revenue opportunities by making the user experience more efficient.

1. Technology Overview

LTU products are based upon the same technological core: the LTU engine. The LTU engine bridges the gap between "low level", i.e. graphical, pixel related information, and "high level", i.e. semantic, user-friendly information.

To achieve this, the LTU engine computes a visual signature for every image that describe its visual content in terms of color, shape, texture and many higher order visual features. These descriptors are also called image DNAs or signatures.

By comparing and analyzing these signatures, the LTU engine can:

  • Find image clones

  • Find images that are visually similar

These functionalities, embedded in all of LTU's products, give businesses the ability to simplify indexing as well as search and retrieval processes within large image databases.

Please contact support@ltutech.com for more information about the LTU engine.

Caution

The core data for LTU engine server is the signature, not the image itself. Once the signature is computed for an image, LTU engine server can, but does not need to store the image.

2. LTU engine server Features

LTU engine server is a server-side software that provides businesses with the ability to search and navigate their visual assets.

The main features of LTU engine server are:

  • Visual navigation: Similarity search within a database to find a set of duplicate, clone or similar images.

  • Image upload: User submission of an image (any format, any source, still image, video or drawing) from its hard drive, real-time indexing and search for visually similar images.

  • Meta-data storage: LTU engine server provides the user with the ability to store information related to every image in its database.

  • Meta-data search: User can search the meta-data for an exact match retrieving the corresponding images.

  • Combined meta-data and visual search: The user can combine meta-data search and image retrieval.

  • Combined semantic/visual search: LTU engine server allows the user to attach and store keywords along with every image. The user can then search for images using keywords only, or perform very efficient search combining visual and semantic queries.

Figure 2.1. LTU engine server

LTU engine server

3. Software Architecture

This section focuses on LTU engine server's server side software.

3.1. Overview: a distributable components architecture

Designed to scale to very large systems and provide high availability, LTU engine server is a set of modular and distributable components. Each set of component is instantiated inside a component server, called a module. The following sections describe each module in detail.

The standard installation of LTU engine server (standalone installation) is hosted on a single machine, with only one instance of the WEKI module. LTU recommends such an installation for most common systems.

For very large systems, LTU recommends the distributed installation of LTU engine server. In order to take advantage of the CPU power of several computers, some modules can be distributed over various machines. To provide redundancy and high availability, one component can be duplicated over several machines.

Caution

Most systems use the standalone installation of LTU engine server. Distributed architecture is usually installed with the help of the LTU consulting group.

Figure 2.2, “Generic Distributed Architecture” summarizes a possible multi-component, distributed LTU engine server system. In a distributed system the roles of each module are:

  • WEKI: Handle HTTP requests, execute scenarios and compute signatures

  • KIMA: cache signatures, perform signature comparisons

Figure 2.2. Generic Distributed Architecture

Generic Distributed Architecture

3.2. The KIMA module

The KIMA module stores signatures in RAM and performs signature comparisons. It is a sort of optimized database cache for image retrieval. In the KIMA modules, the signatures are hosted into repositories.

Signature repositories can be distributed over several KIMA modules. Thus, deploying a large number of KIMA modules makes it possible for the system to deal with very large signature repositories. The KIMA RAM footprint is linearly dependant on the number of signatures in the repository.

Note

On a standalone installation, the KIMA is integrated in the WEKI module. When KIMA is integrated in the WEKI module, not all the features are availables.

3.2.1. Standard distribution

If multiple KIMA modules are installed, each one of them has a unique identification number ranging from 1 to n where n is the total number of KIMA modules. This ID, defined by the administrator, is used to distribute repositories among all existing KIMA modules. Indeed, each repository has a unique internal number automatically assigned by the system.

One KIMA can load one or more repositories.

Example for 9 repositories with internal number from 1 to 9 and 2 KIMA modules: KIMA #1 will load repositories: 1, 3, 5, 7 and 9, while KIMA #2 will load repositories 2, 4, 6 and 8.

3.2.2. Distribution with partitioning of repository(ies)

Partitioning one repository on several KIMA modules, hosted by several machines, has several advantages:

  • Improving the maximum number of signatures in that repository.

  • Reducing the image retrieval response time: signature comparisons are performed in parallel on all KIMA modules.

The partitioning is an override mechanism of the automatic repository deployment explained above and is based on the KIMA ID.

3.3. The WEKI module

The WEKI module has three missions: handle HTTP requests, execute API requests, and compute signatures.

Since WEKI computes signatures, it is responsible for creating and deleting signature files on disk. As signatures are shared with the KIMA modules, KIMA and WEKI modules must have access to a shared storage area, if LTU engine server is distributed over several machines.

Signature computation is CPU intensive. It is necessary to compute signatures for image enrollment into the system or for image search by image upload. The total number of enrollments or searches by upload in a day is thus limited by the CPU power of the system. In order to enhance this power, one can run several WEKI modules on separate machines, and load balance the queries among these modules. There is no limitation on the number of WEKI modules that may be run in an LTU engine server system.

Caution

If using several WEKI modules, the client side load-balancing process that distributes queries among the different WEKI modules has to take session information into account.

4. LTU engine server signatures

When configuring the system, the LTU engine server administrator has to choose the best signature for his images. Indeed, bodies of images can have particularities that should be used to optimize search and navigation in the database.

Caution

Please choose the signature properly. If a misconfiguration were made, any previously added image would have to be re-computed.

LTU engine server analyzes all images at the same size. The bigger this dimension, the longer the CPU time. In order to reduce CPU time, the user can choose to use a smaller processing size, e.g. 192, that will reduce the signature computation CPU time by a factor of 2. Before using smaller signatures, the LTU engine server administrator should check that the quality does not downgrade during retrieval for a specific corpus.

Table 2.1, “LTU engine server signatures” summarizes the various signatures provided by LTU engine server.

Table 2.1. LTU engine server signatures

Signature IdSignature typeWorking image dimensionSize (Bytes)
2photo_medium192x1922976
8dominant colors signature128x128290
60 (60.1.0)local matching256x2562056
60.1.1similar to 60.1.0 , used for fineImageComparison256x2562056
62 (62.1.0)local matching with rotation invariance256x2562056
65 (65.0.0)local matching DNA 2010 with text removal and vertical flip support256x2562056
70 (70.1.0)local matching DNA 2010 with asymetric retrieval512x5124096

Note

A signature type is configured for each repository.

To use fineImageComparison ( signature 60.1.1 ) , original images must be stored.

Chapter 3. Setting Up the Database

LTU engine server needs to have access to a database to start up. LTU engine server 5.0 supports 2 database softwares : Microsoft MsSql and PostGreSql.

1. Microsoft SQL Server 2005

LTU engine server is delivered with scripts for the initialization and administration of the database. You only have to create an user and a database for LTU engine.

Note

If you have to update Microsoft SQL on a LTU engine server anterior to 3.0, please contact LTU support.

2. PostGreSql

Note

IMPORTANT : On linux, using default database settings, the PostGreSQL configuration is automaticly done by the installation script. If you don't need a special configuration, you can skip all this chapter.

  • On linux, LTU engine server requires PostGreSql 8.1 or higher (However, PostGreSql 8.4 or higher is recommended). Please refer to Table 4.3, “Required RPMs on Linux systems” to select linux postgresql related packages to install.

  • On windows, we recommend PostGreSql 9.0.

Note

Database speed greatly depends on the internal index quality. We recommend the user of the PostGreSql command "vacuum" to recalculate indexes once in a while.

Caution

You must be logged as root to perform all the installation tasks.

  1. Initialize databases and configuration files with a specific charset

    When PostGreSql is run for the first time, it initializes the database calling "initdb". If you need the database to support another charset than the default one, you need to call "initdb" first with the right arguments. If you do not need support for special characters, you can jump directly to the next step.

    The initialization must be done like this:

    $ su - postgres
    $ initdb --encoding ISO8859-1

    where "ISO8859-1" can be replace by the charset you need.

    Note

    Contact LTU support for more information about database charsets depending on your PostGreSql version.

  2. Authorize TCP access on PostGreSql

    If you are using PostgreSQL version 8.x or newer, use the following instructions.

    You need to edit PostgreSQL configuration file /var/lib/pgsql/data/postgresql.conf (or /etc/postgresql/<version>/main/postgresql.conf , /etc/postgresql/8.4/main/postgresql.conf for PostgreSQL 8.4).

    Find configuration line that read as follow :

    listen_addresses='localhost'

    Next, set IP address(es) to listen on. You can comma-separated list of addresses. Default is 'localhost', and '*' is all.

    listen_addresses='*'  

    With other version of PostGreSql

    If neither of the above method is possible, contact LTU Technologies support.

  3. Start PostGreSql

    $ /etc/init.d/postgresql start

    This step is important to create the configuration file below.

  4. Authorize the server to access PostGreSql

    LTU engine server needs to access to PostGreSql via TCP connections, you need to explicitly authorize these connections. Edit the file '/var/lib/pgsql/data/pg_hba.conf' (/etc/postgresql/8.3/main/pg_hba.conf for posgresql 8.3 and higher on Debian distribution) and add the authorize line at the end. For instance, to authorize the server "10.0.0.56" to access to PostGreSql, add the following line:

    host all all 10.0.0.56/32 trust

    The general structure is :

    host Database User CIDR-ADDRESS Method

    or

    host Database User IP-address IP-mask Method

    For more details, refer you to http://developer.postgresql.org/pgdocs/postgres/auth-pg-hba-conf.html.

  5. Restart PostGreSql

    $ /etc/init.d/postgresql restart
  6. Create the database user with database creation right

    To create the user "dbuser", use the command "createuser". The following line creates the user.

    $ su - postgres 
    $ createuser -P -A -d dbuser 

    You have to give the user the rights to create databases.

  7. Create the database instance

    To create the database instance "dbname", use the command "createdb". The following line creates the db (assuming that the postgresql server IP address is 127.0.0.1).

    $ createdb -h 127.0.0.1 -U dbuser dbname

    Example :

    $ createdb -h 127.0.0.1 -U my_ltu_dbuser my_ltu_dbname

Chapter 4. Installing or upgrading LTU engine server

The software is provided on our Download Center or in one compressed file named LTU-Engine-Server-5.0-0086.ix86.linux.tgz.

Caution

This version does not support automatic upgrade from previous versions. Nevertheless you can use the database from previous versions (see Section 9.3, “How to use a new LTU engine server version with previous database”).

Prior to installation, make sure that a previous version of LTU engine server or Image Seeker is not installed.

1. System Requirements

This section provides a list of minimum software and hardware requirements for running LTU engine server 5.0.

Table 4.1. LTU engine server system requirements

Operating SystemsCentOS 5, Debian 5, Debian 6, Windows server 2003 and Windows server 2008. Support by default 64bits Operating systems.
Software disk space4 Gbytes on Linux, 8 Gbytes on Windows.
Signature disk spaceThe necessary disk space depends on the signature used and the number of images.
Thumbnail disk spaceThe thumbnail default size is 128x128. These images are around 5 Kbytes (it is possible not to store thumbnails).
Original image disk spaceThe original image size can dramatically vary with their dimension and their format (it is possible not to store original images).
Memory2 Gbytes + necessary memory for signatures.
CPU on Intel arch.Minimum: Intel Pentium IV 2GHz or equivalent. Recommended: Intel Pentium IV 3GHz or equivalent.
DatabasePostGreSql 8.1 or higher ( 8.4 or higher recommended ), Microsoft SQL Server 2005.

Signatures, binary, original and thumbnail images storage location is configurable. For the default image databases, the storage location is done on disk in partition "/var" on Linux or c:/LTU/var. The disk space required depends on the number and the average size of the images, but to give an example, with photo images found on Internet, 4000 images with signatures and thumbnails takes 650 Mbytes on disk.

For LTU engine server distributed architecture, machine sizing depends on how many modules are running and the number of images hosted, in particular:

  • The KIMA modules load signatures in memory. Refer to the previous chapter to size the memory for each host running a KIMA module.

  • The WEKI modules handle HTTP requests and index images. They are more CPU consuming.

Important

Regarding virtualized environment, we support it but only ones running under VM Ware.

2. Supported configurations

The following table shows the list of databases supported by LTU engine server on Windows and Linux with 32 bits and 64 bits for standalone and distributed installation.

Table 4.2. Supported Databases on Linux and Windows

OSStandalone systemDistributed system
Linux 32bitsPostGreSqlPostGreSql
Linux 64bitsPostGreSqlPostGreSql
Windows 32bitsMicrosoft Sql Server, PostGreSqlMicrosoft Sql Server, PostGreSql
Windows 64bitsMicrosoft Sql Server, PostGreSqlMicrosoft Sql Server, PostGreSql

3. Software Requirements on Linux

LTU engine server has been tested on standard "server" installation of Centos and Debian Linux. The exhaustive list of the required files and software packages is in Table 4.3, “Required RPMs on Linux systems”.

Table 4.3. Required RPMs on Linux systems

Distribution and versionRequired RPMsRemarks
CentOS 5 and higherwhich, httpd, mod_ssl, php, gettext, php-xml, compat-libstdc++-296, zip, redhat-lsb, libgfortran44, libgompNeeded packages for the system to work
CentOS 5 and higherperl, perl-XML-SAX, perl-XML-NamespaceSupport, perl-XML-LibXML, perl-HTML-ParserFor the extra perl tools
Debian 5 and higherdebianutils, apache2, libstdc++6, libfreetype6, libx11-6, php5, libapache2-mod-php5, gettext, libgomp1, lsb-core, libgfortran3, lib32gomp1 (ia32-libs in 64bits only)Needed packages for the system to work
Debian 5 and higherperl, libxml-libxml-perl, libcrypt-ssleay-perlFor the extra perl tools
All Linux versionspostgresql ( + postgresql-server for CentOS )If using the database PostGreSql
All Linux versionsunixODBC ( unixodbc-dev for Debian )For distributed mode
All Linux versionspostgresql-odbc ( odbc-postgresql for Debian )For distributed mode with PostGreSql

Use the yum ( aptitude for Debian ) installation system when installing these packages as they may depend on other packages.

Caution

if you install LTU engine on 64bits OS, make sure that the software packages "freetype.i386" and "libX11.i386" are installed on your machine. On 64 bits OS you may need to install the package redhat-lsb-3.1-12.3.EL.el5.centos.i386.

if you are testing LTU engine on Red Hat Enterprise Linux Server 5.X (not officially supported) , you have to update a symbolic link with "ln -s /lib/ld-linux.so.2 /lib/ld-lsb.so.3".

4. Distributed or standalone installation

The first step in the LTU engine server installation process is to choose between a standalone and distributed installation of the software. The standalone installation is suitable for most applications, you should choose the distributed installation in these cases:

  • You need to load a large database of images on one machine.

  • You cannot load the whole database of images on one machine.

  • You want to split the image database over several machines to insure fail-over and load-balancing of your system.

The default parameters of LTU engine server standard distribution are set for distributed installation.

If LTU engine server is to be distributed on several servers, the same software distribution and configuration files must be installed on all servers. The configuration for each machine is different depending on the modules running. Distributed installation and configuration is detailed in Chapter 5, Deploying a distributed LTU engine server system.

5. Upgrading from LTU engine server previous versions

Upgrading from previous version require backing-up the software configuration, the uninstallation of the previous version and the installation of the new version. The database content can be kept, the schema will be upgraded if necessary when installing the new version.

5.1. Database schema upgrade

The database schema upgrade from previous versions is automatic and is done in the database manager tool "dbManager" which is run during the install or can be executed separately.

The tool detects that an upgrade is necessary and automatically upgrade the schema. The upgrade will not alter any of your data but just add column in tables or/and add new tables. The upgrade is done on all repositories present in the database.

5.2. Frontend configuration file upgrade

New versions of LTU engine server may introduce some changes in the XML grammar of the configuration. Before using your old configuration file, check with LTU support team whether you need to change them or not.

6. Installing on Linux

6.1. Pre-installation tasks

Before starting the installation process, make sure you have fulfilled the following tasks.

Database setup

Before running the installation script, you must first setup the database access where LTU data will be stored. Please refer to Chapter 3, Setting Up the Database.

The database configuration needed is:

  • On PostGreSql and MSSQL, you need a user with database creation rights.

The installation program will use these parameters to connect to the database and create the necessary schema.

Software license

The installation program will install the license you provide. Please refer to Section 8, “LTU engine server license” for more details about license information.

For an installation from an archive file downloaded from LTU's client download center, you have to uncompress the archive file in a temporary directory. The archive file contains necessary files for installing LTU engine server as well as standard LTU engine server plug-in files or patches if any.

Note

If LTU engine server is already installed and you want just use a new license, you absolutely must remove old license from the folder "/etc/ltu".

  1. Create a temporary directory

    $ mkdir /tmp/LTUEngineServerInstall
  2. Uncompress archive file in the temporary directory

    On Linux:

    $ cd /tmp/LTUEngineServerInstall
    $ tar -zxvf LTU-Engine-Server-5.0-0086.ix86.linux.tgz

6.2. Installing and configuring LTU engine server

The installation and the configuration of LTU engine server are done with the program "ltu-setup". Once the installation is done, the program may be run several times to configure or re-configure the software (the installation part will be skipped).

The default installation directory is "/home/ltu" on Linux but it can be changed to any shared or local directory. In this section, the software is installed in "/home/ltu".

To start the installation, run ltu-setup.

./ltu-setup
**************************************************************************** 
*                                                                          *
* LTU-Engine Server installation and setup                                 * 
*                                                                          * 
* Copyright (c) 1999-2011 LTU Technologies                                 * 
* All rights reserved                                                      *
*                                                                          * 
**************************************************************************** 
-------------------------------------------------------------------- 
1: Shell libraries 
-------------------------------------------------------------------- 
Loading LTU Technologies main shell library                     [ OK ] 
Loading linux shell library                                     [ OK ] 
Loading database shell library                                  [ OK ] 
Loading LTU Technologies install shell library                  [ OK ] 
Install log file: /tmp/ltu-setup.211204.log 
Refer to this file if you encounter any problem during the installation process. 

The installation program generates a log file (in our example: /tmp/ltu-setup.211204.log). The administrator can refer to this file for installation details and possible errors.

--------------------------------------------------------------------
2: Operating system checks
--------------------------------------------------------------------
System compatibility                                                [ OK ]
Read version file 'VERSION.txt'                                     [ OK ]
Check OS compatibility (VERSION.txt)                                [ OK ]
Checking required packages                                          [ OK ]
Checking optional packages                                          [ OK ]
Checking optional package 'postgresql'                              [ OK ]

First; the system checks the operating system according to system and OS compatibility with LTU engine server 5.0 as well as according to required and optional packages.

---------------------------------------------------------------------
3: Installation
--------------------------------------------------------------------
STATUS: LTU engine server is not installed on your host
Read version file 'VERSION.txt'                                     [ OK ]
    --------------------------------------------------------
    ------ ./VERSION.txt 
       Build generation date : 2011-03-16 01:46:47
       Product               : LTU-Engine-Server 5.0
       Build number          : 0000
       Target system(s)      : CentOS 5.4 ix86, CentOS 5.5 ix86, CentOS 5.3 ix86, Debian 5.0.4 ix86, Debian 5.0.5 ix86, Debian 5.0.6 ix86, Debian 5.0.7 ix86, Debian 5.0.8 ix86, Debian lenny/sid ix86, Debian squeeze/sid ix86
    --------------------------------------------------------
Do you want to install LTU engine server on this host (Y)es/(N)o? [Y]

Second, the program checks if LTU engine server is already installed on the system. If it is not, the program displays the information about the version you are about to install (file "VERSION.txt" from the distribution).

Note

When the installation program asks a question, it may display a default answer, to select this answer, type ENTER. When possible responses are proposed, the only allowed answers (apart from the default value) are surrounded by parenthesis. For example, the yes/no question requires Y or N for answer.

The Linux operating system may be not be officially supported by LTU Technologies , in that case, the install program will stop indicating that the operating is not supported. To bypass this check, you may rerun ltu-setup with the option "--force".

The install directory /home/ltu is where all LTU engine server binary files will be installed. As administrator, make sure that the directory is in a partition with enough disk space (see Table 4.1, “LTU engine server system requirements”).

--------------------------------------------------------------------
3.1: LTU engine server installation
--------------------------------------------------------------------
Creating directories                                                
Unpacking LTU-Engine-Server configuration files                     [ OK ]
Unpacking LTU-Engine-Server binary files                            [ OK ]
Setting installation directory in configuration file                [ OK ]
Copy ltud to /etc/init.d/                                           [ OK ]
Copy mond to /etc/init.d/                                           [ OK ]
Change startup script owner                                         [ OK ]
Register LTU engine server services                                 [ OK ]
Register mon service                                                [ OK ]
Use system apache for admi module                                   [ OK ]
Installation of LTU engine server on this host                      [ OK ]

LTU may have provided patches. In this case, they are installed at this step. In this example, no patch has been provided and thus no patch will be installed.

--------------------------------------------------------------------
4: Patch installation
--------------------------------------------------------------------

Plugin files if any can also be installed. In this example, no plugin file will be installed.

--------------------------------------------------------------------
5: PlugIn installation
--------------------------------------------------------------------

In case of a distributed installation, make sure that the ltu user id will be the same on all servers of the distributed system.

--------------------------------------------------------------------
6: System check
--------------------------------------------------------------------
Deleting group ltu                                                  [ OK ]
Adding user ltu                                                     [ OK ]
Link '/etc/ltu/perl-libs'                                           [ OK ]
Link '/etc/ltu/Smarty'                                              [ OK ]
Create log directory                                                [ OK ]
Create run directory                                                [ OK ]
Create fe directory                                                 [ OK ]
Create fe/templates_c directory                                     [ OK ]
Create fe/views directory                                           [ OK ]
Create fe/datamodels directory                                      [ OK ]
Create fe/images directory                                          [ OK ]
Create fe/exports directory                                         [ OK ]
Do you want to install a license file (Y)es/(N)o? [Y] 

The setup program checks the installation and enters into the license installation process. You may choose to skip the license installation and install it afterwards using "ltu-setup" again or manually by copying the license file in "/etc/ltu" (with ".lic" extension, please refer to Section 8, “LTU engine server license”).

You may install the license from hard drive (option directory). Once you have chosen a license, the installation program asks you to confirm. If you want to install another license at that moment you can answer "A" (for abort), the license installation process will restart from the beginning.

Caution

The program does not check the validity of the license. Please refer to Section 8, “LTU engine server license” for more information.

Installing your license
Select a media < directory,skip > [directory]: directory

In this example, the licence file is located in the directory /home.

Enter the directory  [/tmp]: /home

File list in (/home)

   [ 0 ] : my_license.lic

Choose a number  [0]: 0

The features of the licence will be displayed. You have then to confirm if you want to install the licence.

Install the license (A)bort/(C)ontinue? [C] C

Copying license file to /etc/ltu                                    [ OK ]
Status of Install license                                           [ OK ]
--------------------------------------------------------------------
Installation of LTU-Engine Server on this host                      [ OK ]
--------------------------------------------------------------------

The software installation part is now finished. When running the "ltu-setup" program again, this part will be skipped.

Now starts the LTU engine server configuration part.

--------------------------------------------------------------------
7: LTU-Engine Server configuration
--------------------------------------------------------------------
Do you want to configure LTU-Engine Server (Y)es/(N)o? [Y] Y

You may choose to exit the program now to configure your system manually. It is however recommended to use the installation setup for configuring LTU engine server, like in this example.

The host Internet address must be the fully qualified Internet name of the host. This name must be understandable by client hosts that will use LTU engine server (do not put addresses like "localhost" or "127.0.0.1").

The host internet address can be the host name or the IP address.
It is used for HTTP references in LTU-Engine Server administration interface
and application API and must therefore be accessible by client hosts.
Enter this host internet address  [myhost.mydomain.com]: myhost.mydomain.com

You may also choose at this moment to secure the access to web interface by SSL.

Setting the host internet address in global configuration file      [ OK ]
The web-based graphic user interface is not secured with SSL.
Secure the web interface access with SSL (Y)es/(N)o? [N] N
Disable SSL for web interface access                                [ OK ]

We configure the system to run a KIMA, a WEKI and an ADMI on this host. For more information about installing a distributed system see Chapter 5, Deploying a distributed LTU engine server system.

8: LTU-Engine Server distribution setup
--------------------------------------------------------------------
--------------------------------------------------------------------
8.1: Distribution configuration
--------------------------------------------------------------------
The module KIMA is activated on this host
Do you want to run a KIMA on this host (Y)es/(N)o? [Y] 
Enter KIMA Id  [1]: 
The module WEKI is activated on this host
Do you want to run a WEKI on this host (Y)es/(N)o? [Y] 
The module ADMI is activated on this host
Do you want to run a ADMI on this host (Y)es/(N)o? [Y]

We use default WEKI and/or KIMA services capacity.

--------------------------------------------------------------------
9: LTU engine system capacity configuration
--------------------------------------------------------------------
Checking allowed cpu capacity
Do you want to configure WEKI and/or KIMA services capacity ? (Y)es/(N)o? [N]

License server is now activated.

--------------------------------------------------------------------
10: LTU License server setup
--------------------------------------------------------------------

License server activated

The program now enters into the database configuration process, by default, it creates a local PostgreSQL database instance.

If you answer yes to "edit the database parameters", and no to "use the default database storage", you have to provide the database parameters (the database name, its address, its port, the login name and the login password). When running the installation program the first time, the default values start with "EDIT::..." for editing convenience otherwise the program shows you the current database settings and asks if you want to change them.

--------------------------------------------------------------------
11: LTU engine server database setup
--------------------------------------------------------------------
 -- Default database storage with the following parameters:
  -   Database user login     : ltu_user
  -   Database user password  : ltu_password

Do you want to edit the database parameters (Y)es/(N)o? [N] Y
Do you want to use the default database storage (Y)es/(N)o? [Y] N
Database module in global configuration file                        [ OK ]
Enter the database address  [127.0.0.1]: 
Enter the database instance name  [ltu_engine]: 
Enter the database port  [5432]: 
Enter the database login name  [ltu_user]: 
Enter the database login password  [ltu_password]: 
 -- Current Database Parameters
  -  Type : POSTGRESQL
  -   Database address        : 127.0.0.1
  -   Database name           : ltu_engine
  -   Database port           : 5432
  -   Database user login     : ltu_user
  -   Database user password  : ltu_password


Confirm parameters (A)bort/(C)ontinue? [C] 
Checking postgresql client binaries                                 [ OK ]
Database access (Version: 8.1.22)                                   [ OK ]
Checking required packages                                          [ OK ]
Change ODBC PostGreSql driver                                       [ OK ]

After you have confirmed the database parameters, the installation program checks that it can connect to the database. In case, the installation program fails to connect to the database you be asked to enter again the database parameters or to exit the installation process.

After the connection to the database is checked, the installation runs the console tool 'db-manager' that will help install the right schema template corresponding to your needs. The administrator may run this tool at any time after installation (c.f. Chapter 6, Administrating LTU engine server).

When the database schema is empty, the tool creates the common tables for all image databases (i.e. repositories).

Launching image database manager...

  ******************************************************************************
  * Image Database Manager - Version 2.0                                       *
  * LTU Technologies - Copyright (c) 1999-2010                                 *
  * All rights reserved                                                        *
  ******************************************************************************

 checking database content...
    ok
 adding/updating host configuration...
    ok
 initializing the image database status...
    from the database
    from the front-end configuration
    from the database sql files
 updating the status of all databases 
 initializing the repository skeleton list...    ok
 initializing the signatures configurations list...    ok
Change current partition string '', enter a new partition string or 'cancel': 
    new partition string to ''. (A)bort/(C)ontinue? [C]

The program shows you the available image repositories and their status on the system. When installing for the first time, the program shows an empty list of image repositories and the actions the administrator can do. In the fisrt time, Enter '1' to create a new repository.

Id : repository identifier

Desc : repository description

Fe : front-end, can take the value 'Y' (actived) or 'N' (desactived)

Db : Database

Ba : backend, can take the value 'Y' (actived) or 'N' (desactived)

Nb Img : number of images stored in the repository

Signature : identifier of the signature used

 ====================== Available image repositories ====================== 
     Id    Desc                   Fe   Db      Ba      Nb Img  Signature 
     --    ----                   --   --      --      ------  --------- 

 * Choices
 0/ Quit
 1/ Create a new repository
 2/ Select/Unselect one repository
 3/ Enable selected repositories in frontend and backend
 4/ Disable selected repositories in frontend and backend
 5/ Uninstall selected repositories in database and in frontend
 6/ [ADVANCED CONFIGURATION] Change the signature type of the selected repositories
 7/ [ADVANCED CONFIGURATION] Change the search signature type of the selected repositories
 8/ Change the repository partition string ''

Your choice ? 1

In the screen below, the administrator can choose between seven repositories templates.

After selecting a repository template, you must complete all parameters related to this new repository (ID, repository name,....).

     ========== Available repository templates ==========

   ID   Name
   --   ----
   1   Color Query
   2   Matching - Fine Image Comparison
   3   Matching - Sig60 - does not support rotation
   4   Matching - Sig62 - supports rotation
   5   Matching - Sig65 - media monitoring
   6   Matching - Sig70 - asymetric retrieval
   7   Similarity - Sig02
 
Choose a repository template : 3
Choose your repository ID [1]: 
Choose your repository name [rep1]: repository1
Choose your repository title in English [Repository 1]: english_title
Choose your repository title in French [Repository 1]: titre_francais
Choose your repository description in English : english_description
Choose your repository description in French : description_francaise


*** Your duplication parameters :

Repository templates = Matching - Sig60 - does not support rotation
Your repository name = repository1
Your repository id = 1
Your repository English title = english_title
Your repository French title = titre_francais
Your repository English description = english_description
Your repository French description = description_francaise


Are your parameters OK ? (A)bort/(C)ontinue? [C]

Now that you have entered all parameters, you can enter 'yes' to install the image database.

/etc/ltu/admi/skeletons/Matching - Sig60 - does not support rotation/admi/databases/_REPNAME_
/etc/ltu/admi/databases
/etc/ltu/admi/skeletons/Matching - Sig60 - does not support rotation/sql/build/postgresql/rep_ID_
/home/ltu/tools/database/build/postgresql
 initializing the image database status...
    from the database
    from the front-end configuration
    from the database sql files
 updating the status of all databases...    ok
 installing image database with Id '1'...
    create database tables and insert default values...
       run sql file 'rep1_ltu_tables.sql'
       run sql file 'rep1_ltu_values.sql'
       run sql file 'rep1_common_entity_tables.sql'
       run sql file 'rep1_entity_tables.sql'
       run sql file 'rep1_common_entity_values.sql'
       ok
    enabling image database with Id '1' in frontend...
       ok
       creating binary directory '/var/ltu/rep1/binary/'...
         ok
       creating image directory '/var/ltu/rep1/images/'...
         ok
 WARNING: you need to restart manually the services. [Press Return to continue]

====================== Available image repositories ====================== 
     Id    Desc                   Fe    Db      Ba      Nb Img  Signature 
     --    ----                   --    --      --      ------  --------- 
( )  1     english_title          Y     Ok      Y       0       60       

 * Choices
 0/ Quit
 1/ Create a new repository
 2/ Select/Unselect one repository
 3/ Enable selected repositories in frontend and backend
 4/ Disable selected repositories in frontend and backend
 5/ Uninstall selected repositories in database and in frontend
 6/ [ADVANCED CONFIGURATION] Change the signature type of the selected repositories
 7/ [ADVANCED CONFIGURATION] Change the search signature type of the selected repositories

Your choice ? 0

 exiting...

During post installation, scripts can be included in the installation procedure in order to finalize the configuration of a plugin.

--------------------------------------------------------------------
12: Post Installation
--------------------------------------------------------------------

The installation program finally tries to start LTU engine server and verifies if all modules are running.

-------------------------------------------------------------------- 
13: Installation finished 
-------------------------------------------------------------------- 
Do you want to start LTU-Engine Server (Y)es/(N)o? [Y] Y 
Restarting LTU-Engine Server                                          [ OK ] 
Status LTU-Engine Server                                              [ OK ] 
-------------------------------------------------------------------- 

STATUS: HOST your_hostname INSTALLATION ENDED SUCCESSFULLY

LTU engine server is now running on your system. The next step is to configure the Internet based interface to give access to content administration users and search users. Section 4, “Web-based administration interface” explains you how to do so.

6.3. Configure syslog

After LTU engine server installation, you can configure syslog logging for kima, weki and jni.

  • To activate syslog logging for kima, update logger.syslog.bEnable = False to logger.syslog.bEnable = True in /etc/ltu/kima-logging.conf.

  • To activate syslog logging for weki, uncomment #handlers= com.ltu.utils.logging.SyslogHandler in /etc/ltu/weki.logging.properties.

  • To activate syslog logging for jni, update logger.syslog.bEnable = False to logger.syslog.bEnable = True in /etc/ltu/ltujni-logging.conf.

You can configure extra syslog parameters in the file listed above as standard syslog usage. For more information, please consult syslog documentation.

Note : by default, the syslog messages will be found in /var/log/messages.

7. Installing on Windows

Note: If using the database MS SQL Server 2005, please install SQL Server Service Pack 2 on the windows machine.

7.1. Pre-installation tasks

LTU engine server has been tested on Windows Server 2003 and Windows Server 2008. LTU engine server does not require any third party software.

LTU engine server requires access to a database. The database user must have rights to create tables in the database. Thus, before running the installation script, ensure that a database has been set up.

LTU engine server 5.0 supports 2 types of database software: PostGreSql 9.0 and Microsoft SQL Server 2005. Prior to installation, gather the following information about the database:

  • Type of database (PostGreSql or MsSql)

  • Address of the database server

  • Name of the database instance

  • Port of the database server: confirm the default values suggested by the installation setup (step 5)

  • Name of the database user

  • Password of the user

7.2. Installing and configuring LTU engine server

The default installation directory is "c:\LTU" on Windows but it can be changed to any local directory. In this section, the software is installed in "c:\LTU".

Note

If LTU engine server is already installed and you want just use a new license, you absolutely must remove old license from the folder "c:\LTU\etc".

  1. Unzip the archive LTU-Engine-Server-5.0-0086.ix86.win.zip into the directory c:\ .

  2. On Windows 64-bits run c:/LTU/tools/install/vcredist_x86.exe and c:/LTU/tools/install/vcredist_x64.exe. If you are running a 32-bits Windows run c:/LTU/tools/install/vcredist_x86.exe only.

  3. Reboot the machine

  4. Copy the specific following libraries "msvcp90.dll", "msvcr90.dll" and "mfc90.dll" from c:\LTU\ltuSystem\native\win32 to c:\WINDOWS\system32. This step may not be necessary depending on the machine configuration.

  5. Install the .NET Framework 3.5 Service pack 1 from c:\LTU\tools\install (for Windows server 2008, use Server Manager/Features/Add Features/.NET framework 3.5.1).

  6. Copy the LTU engine server license file into c:\LTU\etc\ltu

    The license file name should have the extension ".lic". Please refer to Section 8, “LTU engine server license” for more details about license information.

  7. Edit c:/LTU/etc/ltu/global.bat and change the following options:

    Parameter nameParameter content
    DATABASEType of database used: MSSQL or POSTGRESQL
    DATABASE_ADDRIP address of the database server
    DATABASE_NAMEDatabase schema name
    DATABASE_PORTDatabase server listening port. Default values are 5432 for Postgresql and 1433 for MsSql.
    DATABASE_LOGINNAMEDatabase user login name
    DATABASE_LOGINPWDDatabase user password
    LMHOSTIDMAC address of the machine without the "-" characters (e.g. 00B0D03BE90E)
    KIMA_64Use 64bits Kima (1 by default, set to 0 to use 32bits Kima)

    You may also increase the amount of memory allowed by the signature cache module (WEKI). To increase the amount of memory, simply increase the value of the parameter WEKI_MEM_SIZE. The default value is 800 MB. This value should not be higher than 2048 MB.

  8. Kima module access to the database with ODBC system, to install and configure ODBC, please refer to the windows Section 3.1, “Configure ODBC access to the database”.

  9. In a command prompt window, change directory to c:\LTU\tools\install and run db-manager.bat

    0/ Quit

    1/ Create a new repository

    2/ Selected/Unselected one repository

    3/ Enable selected repositories in front-end and backend

    4/ Disable selected repositories in front-end and backend

    5/ Uninstall selected repositories in database and in front-en

    Db-manager tool is used in the same way as on Linux. Please refer to Section 5.2 for more details about using db-manager.

LTU engine server is now installed on your machine. You should now manually start the LTU engine server modules to check that the server is correctly configured. Open a new command line console window for each.

CHECKING FLXM CONFIGURATION To check is flxm is configured correclty, run the following

c:\LTU\bin\flxm-startup.bat

For more information, please see Section 8, “LTU engine server license”.

CHECKING ADMI CONFIGURATION To check if admi is configured correctly, run the following:

c:\LTU\bin\admi-startup.bat

If successful the command will say '+ Run Apache 2' and not return or show errors. Leave this window open and proceed to the next test.

CHECKING WEKI CONFIGURATION To check if the weki is configured correctly, run the following:

c:\LTU\bin\weki-startup.bat

If successful the window will scroll pages of log messages and complete by returning with no errors to the command line.

RUNNING THE LTU ENGINE SERVER One you have verified that admi and weki run and are installed, you are now ready to start LTU engine server.

START VIA CONSOLE To run LTU engine server via the console will open two windows. The server will run for as long as these two windows are open, closing the windows will stop the server. Run the following:

c:\LTU\bin\ltu-start-console.bat

START VIA WINDOWS SERVICE To install LTU engine server as a Windows service. You only need to be install the services once. You may have problems if you try to install services when they already exist and are running. After a successful install there will be four new services 'LTUADMI', 'LTUWEKI', 'LTUFLXM' and 'LTUWEKI'. To install run the following:

 c:\LTU\tools\install\ltu-install-services.bat

This script should install the four Windows services named "LTUADMI (Apache 2)", "LTUKIMA", "LTUFLXM" and "LTUWEKI (Tomcat server 5)".

UNINSTALL THE LTU WINDOWS SERVICES To uninstall the services, run the following:

 c:\LTU\tools\install\ltu-uninstall-services.bat

For log rotation use log-kima-rotation.bat that you have to add in a new scheduled task (every month for example). Logs rotated with 10 logs files.

8. LTU engine server license

LTU engine server software is protected by a license. Every machine that is part of the LTU engine server system must have a license. Ask your LTU software reseller for a valid license or ask LTU Technologies directly by sending an email to license-request@LTUtech.com.

8.1. Licensing mechanism

In the case of a missing or invalid license file, LTU engine server License Server fails to start. This is the error you will get on Linux:

$ /etc/init.d/ltud status 
License Server (23129)                         [ FAILED ]
KImage Repository Module (23160)               [ OK ] 
Web KImager Module (23487)                     [ OK ] 
ADMInistration Module (23501)                  [ OK ] 

Caution

The license server is activated by default. It is defined on linux by the FLXM_RUN in /etc/ltu/local.conf and on windows by FLXM_RUN in "[c:/LTU/]etc/ltu/global.bat".

8.2. Getting information for an LTU engine server system

For each machine that is a part of your LTU engine server system, you will need to have a valid license. On Linux and Windows, get the MAC-Ethernet address, the number of CPUs and the hostname.

8.2.1. Getting the MAC address on Linux

If LTU engine server is already installed, you can use a local program to directly get the host Identifier information:

$ [LTU_HOME]/bin/linuxCO46/lmhostid 
Lmhostid - Copyright (C) 1989-2001 Globetrotter Software, Inc. 
The FLEXlm host ID of this machine is "00B0D0384919" 

On Linux, to obtain the MAC address of the primary network card:

$ ifconfig -a 
eth0 Link encap:Ethernet HWaddr 00:B0:D0:38:49:19 
     inet addr:10.1.10.24 Bcast:10.1.10.255 Mask:255.255.255.0
     UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
     RX packets:54939 errors:0 dropped:0 overruns:1 frame:0
     TX packets:21471 errors:0 dropped:0 overruns:0 carrier:1
     collisions:0 txqueuelen:100
     Interrupt:5 Base address:0xec00 
lo   Link encap:Local Loopback
     inet addr:127.0.0.1 Mask:255.0.0.0
     UP LOOPBACK RUNNING MTU:16436 Metric:1
     RX packets:114 errors:0 dropped:0 overruns:0 frame:0
     TX packets:114 errors:0 dropped:0 overruns:0 carrier:0
     collisions:0 txqueuelen:0

The MAC address is "00:B0:D0:38:49:19".

8.2.1.1. Getting the MAC address on Windows

To obtain the MAC address of your machine:

  1. Open a command prompt in Windows

  2. Type:

    ipconfig /all
  3. All information about your network connection is displayed, including the physical address of your network interface (the MAC address):

    Physical address     : 00-04-76-D8-30-BB
  4. Provide this MAC address to LTU technologies to obtain a valid LTU engine server license.

8.2.2. Getting the number of CPU(s) of a machine

On Linux, to obtain the number of CPU:

$ grep processor /proc/cpuinfo | wc -l 
2

On windows, to obtain the number of CPU, you will need to open the machine property window.

8.2.3. Getting the host name of a machine on Linux

To obtain the host name of your machine:

$ hostname
my_server

8.3. The FLXM module (license(s) server)

Only one FLXM module must be started for an LTU engine server system (see Chapter 7, Configuring LTU engine server on how to start the license server). This process (called lmgrd located in the bin directory) is a lightweight process using almost no resource. By default it uses the port 27000.

8.4. Requesting a license

8.4.1. Mono host installation (standard installation)

Get all requested information from the unique host and ask your LTU software reseller for a valid license or ask LTU Technologies directly by sending an email to license-request@LTUtech.com.

8.4.2. Multi hosts installation (custom installation)

Get all requested information from all the hosts of your system, specify the host name for the FLXM module and ask your LTU software reseller for a valid license or ask LTU Technologies directly at support@LTUtech.com.

8.5. Installing the license file

The license file has a ".lic" extension and have to be placed in "/etc/ltu/". In case of a multi host installation, the license file must be only installed on the FLXM module host.

Note

Do not move/remove the license file, the software checks it periodically.

Format of a simple license file:

SERVER MY_SERVER 005056B50014 
VENDOR ltutech 
FEATURE 60 ltutech 1.000 31-jan-2012 uncounted HOSTID=005056b50014 \
  TS_OK SIGN="0087 973F 44F0 CDD8 97AE 8C5A 0E85 E300 A2AC BAE2 \
  A490 E320 FBEB F75A 7BF9"
FEATURE indexation ltutech 1.000 31-jan-2012 uncounted \
  HOSTID=005056b50014 TS_OK SIGN="002E 60FF B711 0D24 015B 23D0 \
  8A09 3000 5A83 F74A 97BC 09DB CE1B 8B06 611A"
FEATURE recognition ltutech 1.000 31-jan-2012 uncounted \
  HOSTID=005056b50014 TS_OK SIGN="0084 C4F2 0CDF BC10 561A 9930 \
  FEEB 2D00 5DC4 DF52 5E05 5320 16F5 5E25 AF5F"
FEATURE retrieval ltutech 1.000 31-jan-2012 uncounted \
  HOSTID=005056b50014 TS_OK SIGN="00B7 9FD4 707A 37ED CB20 D6AA \
  7BEE AD00 866D 1CB4 EE65 306D 1432 E058 1A3F"
FEATURE ltu-kima-server ltutech 1.000 31-jan-2012 uncounted \
  HOSTID=005056b50014 TS_OK SIGN="0078 CDEF A771 3296 1418 C390 \
  0A42 8A00 924D 2720 CEE5 0E5E B951 322F CB14"
FEATURE ltu-cpu-capacity ltutech 1.000 31-jan-2012 2 \
  HOSTID=005056b50014 TS_OK SIGN="0088 71BF 565E 879C 6C40 B120 \
  43A2 3000 D37F 0179 BEF9 2F07 DC9F F721 9589"
### LTU engine server: Database limitation
#### 10k images
FEATURE db-nbimages-add-limit ltutech 1.000 31-jan-2012 10000 \
  HOSTID=005056b50014 TS_OK SIGN="004F 04BA 3113 9979 92BD 8EB7 \
  968F F400 8B9F 41C0 4922 2855 A3A1 DA8E DD64"
FEATURE db-nbimages-system-limit ltutech 1.000 31-jan-2012 10200 \
  HOSTID=005056b50014 TS_OK SIGN="0046 87D0 3B63 B713 1B89 F379 \
  E487 E400 CAC2 D600 82A7 676D C9D2 C55A 49E8"

Where "ltu-cpu-capacity" is the number of CPU allowed on the global system. In case the number of CPU is not limited, this value is "uncounted". The license contains several features (each is a line starting with the word FEATURE). The features contain what LTU engine server can do. The main features are: "indexation" and "retrieval" and the signature identifier allowed.

8.6. Log messages on license check

All modules check the license server for a valid license at startup. Following are log message examples (c:\LTU\var\log\ltu\flxm.log by default) :

No license

license manager: can't initialize:Cannot find license file.
The license files (or license server system network addresses) attempted are
listed below.  Use LM_LICENSE_FILE to use a different license file,
or contact your software provider for a license file.
Filename: "/etc/ltu/*.lic"
License Path: "/etc/ltu/*.lic"
FLEXnet Licensing error:-1,359
System Error:2 No such file or directory
For further information, refer to the FLEXnet Licensing documentation,available at "www.acresso.com".
Using license file "/etc/ltu/*.lic"

Invalid license

Std out log: Mar 10 10:03:23 ERROR Invalid (inconsistent) license key 
 The license-key and data for the feature do not match. 
 This usually happens when a license file has been altered 
Feature: ltu_feature_1 
License path: /etc/ltu/my_license.lic 

Valid license

Std out log: Mar 10 13:13:30 INFO Checking license

9. Maintenance

Please do not hesitate to contact our support team if you face any issue or if you have any question regarding procedures described in this chapter.

9.1. How to backup LTU engine server

First you need to backup your database, if you are using the default one please follow the procedure, please replace name in capital letter by your parameter.

  • Connect to a console and use this command:

    • On linux:

      go to the directory where you want to backup

      pg_dump -U ltu_user ltu_engine --file backup_ltu_engine_db.dump --clean

    • On windows:

      cd "C:\Program Files\PostgreSQL\9.0\bin"

      pg_dump.exe -f c:\YOUR_BACKUP_DIRECTORY\backup_ltu_engine_db.dump --clean -U YOUR_USERNAME DATABASE_NAME

  • Then backup your directories:

    • On linux:

      /var/ltu

      /etc/ltu

    • On windows:

      c:\LTU\etc\ltu

      c:\LTU\var\ltu

9.2. How to restore LTU engine server

  • Unpack your archive with the var/ltu and etc/ltu folders in / for linux and in c:\LTU for windows.

  • Connect to a console and use this command:

    • On linux:

      psql -U ltu_user -d ltu_engine -f /YOUR_BACKUP_DIRECTORY/backup_ltu_engine_db.dump

    • On windows:

      cd "C:\Program Files\PostgreSQL\9.0\bin"

      psql.exe -U YOUR_USERNAME DATABASE_NAME <c:\YOUR_BACKUP_DIRECTORY\backup_ltu_engine_db.dump

9.3. How to use a new LTU engine server version with previous database

To use an existing LTU engine server database with a new LTU engine server version, you have to :

Chapter 5. Deploying a distributed LTU engine server system

Any LTU engine server system can be distributed over several hosts, so as to build a highly available, scalable system. This section describes the step-by-step "how to" deploy distributed LTU engine server. See Chapter 2, LTU engine server Overview for a general overview of the distributed LTU engine server system.

1. Architecture design

Distributed LTU engine server requires 3 kinds of modules:

  • One or more KIMA modules

  • One or more WEKI modules

  • One or more ADMI modules

1.1. ADMI module

Many ADMI modules can be installed on several machines to build a load balanced / fail over web configuration. In order to be able to share correctly the data between the ADMI modules, this configuration requires an NFS file server to be set up.

There are many ways to achieve a load balancing/fail over solution for web servers. Hardware will be the most reliable, software may be less cost-consuming. There are only a few rules to configure ADMI modules so that they can share data with other ADMI, they are in the following chapter.

1.2. WEKI module

They use CPU power quite a lot, but do not require a lot of RAM. Deploying a large number of load-balanced WEKI modules will enable a great number of client to do queries simultaneously.

1.3. KIMA module

They require fast and large RAM and CPU power.

The number of signatures hosted by a KIMA module is limited by the available RAM size. Since 4Gb is the maximum RAM that can handle a process on a 32bits machine, you should use a 64bits operating system to overpass this limit.

1.4. Designing the architecture

Note

It is recommended to refer to LTU consulting team to validate your distributed LTU engine server system architecture.

Designing the architecture means choosing what modules to install on what host. The constraints on the architecture design are the following:

  • There are as many WEKIs and KIMAs as possible

  • The total amount of memory allocated to KIMA modules is enough to cache all the signatures of all the repositories

Note

One machine can host one instance of WEKI and KIMA modules simultaneously. One server cannot host several instances of the same module.

2. Deploying and installing all the hosts on Linux

You need to deploy and install LTU engine server on all the hosts of the architecture. To do so, on every host, follow the instruction from Chapter 4, Installing or upgrading LTU engine server, and enter the distribution configuration part.

Now, you have to configure every host according to the architecture design elaborated above. This section describes how to configure each host step by step. In the installation detailed above, we are installing a host with all components on a machine and another machine with only a KIMA running. Each repository will be hosted on each of the two KIMA modules according to its identification number.

See Chapter 7, Configuring LTU engine server for details on all parameters that are used in this section.

Caution

If you plan to store the images and signatures on disk. You should make sure that the 'ltu' user created on every machine of the system has the same Id so that all module can access the files.

2.1. Configure ODBC access to the database

Kima module access to the database with ODBC system. Using default installation settings, you don't need to manually configure postgresql odbc. Using specific configuration, you will need to install an ODBC client and ODBC drivers. On linux, unixODBC (http://www.unixodbc.org/) manage PostGreSql database. A package called unixODBC is available for redhat distributions ( CentOS ). See Table 4.3, “Required RPMs on Linux systems” for more information on required odbc packages.

After installing this package, follow those instructions :

2.1.1. PostGreSql

Install the package called postgresql-odbc and then edit those files :

/etc/odbcinst.ini

[PostgreSQL]
Description                   = ODBC for PostgreSQL
Driver                             = /usr/lib/psqlodbc.so
Setup                              = /usr/lib/libodbcpsqlS.so
FileUsage                      = 1

/etc/odbc.ini

[LTU]
Description                  = PostgreSQL
Driver                            = PostgreSQL
Trace                             = No
TraceFile                      =
Database                     = EDIT::Your database name
Servername                = EDIT::Your server address
Username                    = EDIT::Your login
Password                     = EDIT::Your password
Port                                = EDIT::Your port (by default : 5432)
Protocol                        = 6.4
ReadOnly                     = No
RowVersioning           = No
ShowSystemTables  = No
ShowOidColumn        = No
FakeOidIndex             = No
ConnSettings             =

2.2. KIMA configuration

The KIMA module configuration through the installation process is described below. The corresponding configuration parameters are explained after.

--------------------------------------------------------------------
8.2: Distribution configuration
--------------------------------------------------------------------
The module KIMA is not activated on this host
Do you want to run a KIMA on this host (Y)es/(N)o? [N] Y
Enter KIMA Id  [1]: 1

Each KIMA server has an ID, that is set in "/etc/ltu/local.conf". To set a KIMA's ID to N, make sure the KIMA_ID variable is set to N in this file: KIMA_ID=N.

Note

KIMA IDs are integers between 1 and N, if N is the total number of KIMA modules in the system.

2.3. ADMI configuration

2.3.1. Distribution of ADMI modules

2.3.1.1. Files and directories that must be on the NFS server :

Some files must be shared by all ADMI modules : place them on the NFS server and then replace them on each ADMI server by symbolic links pointing on NFS server.

Note that the user "ltu" from group "ltu" created on each machine will have to share the same userid to be able to access in read/write mode on all shared files.

The files and directories to be shared are listed below :

  • /etc/ltu/admi/admi_config.xml

  • /etc/ltu/admi/admi_tree.xml, /etc/ltu/admi/admi_tree_translator.xml

  • /etc/ltu/admi/frontend.conf

  • /etc/ltu/admi/databases

  • /etc/ltu/admi/dtd

  • /etc/ltu/admi/plugins

  • /etc/ltu/admi/rights

  • /etc/ltu/admi/system

  • VARS_DIR/users where VAR_DIR is the directory storing users data and images (/var/ltu by default)

  • INSTALL_DIR/ltuSystem/http_data/htdocs/tmp where INSTALL_DIR is the installation directory.

2.3.1.2. PHP session sharing

There may be two ways of configuring a load balancer application : either it will redirect randomly any request to one of the ADMI, either it can analyze the request header, get the PHP session id and depending on it redirect to the given ADMI that has generated this ID.

In that last case, once a user is logged on a given ADMI, he will always stay on it, and there is no need for the PHP servers to share their session data with other.

In the first case however, once a user is logged on an ADMI server, he can then be redirected during the same session on any ADMI : they have to share their session data with the originating server. Follow the steps below to achieve this :

  • Create a directory on the NFS server that can be accessed by all ADMI

  • In the /etc/ltu/admi/php.ini file of each ADMI server, change the property session.save_path to this shared directory.

2.3.1.3. Common Configuration for all ADMIs

All ADMI modules must be configured to point on the one and only WEKI module of the application. There are to location to change :

  • In each /etc/ltu/admi/databases/rep_name/repXXX_config.xml, set the "host" property of the SERVER element to the address of the WEKI server :

    <SERVER apiVersion="4.0" host="WEKI_SERVER" logLevel="-1" port="8090" retry="0" timeout="0"/>
  • In the /etc/ltu/admi/httpd/httpd_1.3.conf and httpd_2.0.conf of each ADMI, change the proxy setting of Apache to redirect all LTU engine server API queries on the WEKI :

    ProxyPass /iseeker/ http://WEKI_SERVER:8090/iseeker/
    ProxyPass /ifilter/ http://WEKI_SERVER:8090/ifilter/
    ProxyPass /common/ http://WEKI_SERVER:8090/common/

2.3.2. Tested Configuration

The following load balanced configuration was tested (see figure below). It does not implement fail-over, but was enough to test the sharing of data between many deployed ADMI module. Thus, using any solution (hardware or software) to implement a load balance and/or fail-over solution will work.

Configuration :

  • 1 SQUID Server : squid is a common HTTP accelerator and proxy cache for web applications. It is used here to force the DNS client to ask the round robin every minute. That way; nearly every queries are directed on a different ADMI server.

  • 1 DNS Round Robin : DNS server serving many I.P. addresses for one host name

  • 1 NFS Server : The file sharing server

  • N ADMI's : many ADMI modules, sharing some data via the NFS server.

  • 1 WEKI : in this version, it is not possible to use more than one WEKI module.

Figure 5.1. Test Configuration

Test Configuration

2.4. Module activation and repository partitioning settings

In the 8.1 configuration part, you can select which KIMA, WEKI and ADMI services you want to run on this host.

--------------------------------------------------------------------
8.1: Distribution configuration
--------------------------------------------------------------------
The module KIMA is activated on this host
Do you want to run a KIMA on this host (Y)es/(N)o? [Y] 
Enter KIMA Id  [1]: 
The module WEKI is activated on this host
Do you want to run a WEKI on this host (Y)es/(N)o? [Y] 
The module ADMI is activated on this host
Do you want to run a ADMI on this host (Y)es/(N)o? [Y]

During LTU engine server database setup for a distributed installation, you can specify a partitioning string.

Launching image database manager...

  ******************************************************************************
  * Image Database Manager - Version 2.0                                       *
  * LTU Technologies - Copyright (c) 1999-2010                                 *
  * All rights reserved                                                        *
  ******************************************************************************

 checking database content...
    ok
 adding/updating host configuration...
    ok
 initializing the image database status...
    from the database
    from the front-end configuration
    from the database sql files
 updating the status of all databases 
 initializing the repository skeleton list...    ok
 initializing the signatures configurations list...    ok
Change current partition string '', enter a new partition string or 'cancel': 
    new partition string to ''. (A)bort/(C)ontinue? [C]

The partitioning string should have the following structure:

REPOSITORY_PARTITIONING="REP_ID:KIMA_ID1,...,KIMA_IDN;REP_ID:KIMA_ID1,...,KIMA_IDN;..."

Each token of that string, delimited by a ';' character, sets a repository to be split on one or several KIMA servers. The repository "REP_ID" will be split on KIMA servers with IDs KIMA_ID1 to KIMA_IDN.

REP_IDs are the real IDs of the repositories in the database. KIMA_ID is an integer between 1 and N, where N is the number of KIMA servers.

In order to partition repository 3 on KIMA servers 1, 2 and 3, the REPOSITORY_PARTITIONING string should look like:

REPOSITORY_PARTITIONING="...,3:1,2,3;..." 
Launching image database manager...

  *******************************************************************************
  * Image Database Manager - Version 2.0                                        *
  * LTU Technologies  - Copyright (c) 1999-2010                                 *
  * All rights reserved                                                         *
  *******************************************************************************

 checking database content...
    no LTU schema
 creating the LTU schema...
    ok
 adding/updating host configuration...
    ok
 initializing the image database status...
    from the database
    from the front-end configuration
    from the database sql filesEd
 updating the status of all databases
Network interface: eth0/eth0 addresses [ fe80:0:0:0:214:22ff:fe42:dd75%2/fe80:0:0:0:214:22ff:fe42:dd75%2/fe80:0:0:0:214:22ff:fe42:dd75%2 arcueil.ltutech.com/10.1.10.151/arcueil.ltutech.com ]
Network interface: lo/lo addresses [ 0:0:0:0:0:0:0:1%1/0:0:0:0:0:0:0:1%1/0:0:0:0:0:0:0:1%1 localhost.localdomain/127.0.0.1/localhost.localdomain ]
Local host address: arcueil.ltutech.com/10.1.10.151/arcueil.ltutech.com
Change current partition string '', enter a new partition string or cancel:

Example: The repositories 10, 5, 4 and 1 are split upon 2 Kimas, Kima 1 and Kima 2:

Change current partition string '', enter a new partition string or cancel: 10:1,2;5:1,2;4:1,2;1:1;2;

Note

Each host has to know what modules it is supposed to run. This is directly configured with "ltu-setup". You may also edit the configuration file "/etc/ltu/local.conf" of every host, and set the appropriate variables to 0 or 1 (run or do not run). Module variables are: WEKI_RUN, KIMA_RUN, FLXM_RUN, ADMI_RUN.

3. Deploying and installing all the hosts on Windows

The distributed system installation is described in this chapter. You must repeat the following on each machine which is part of the distributed system. Before changing from a standalone installation to a distributed one, make sure that all of the LTU engine server Windows services are uninstalled.

3.1. Configure ODBC access to the database

Kima module access to the database with ODBC system.

3.1.1. PostGreSQL

Install PostGreSQL Odbc 64bits driver from c:\LTU\tools\install.

Edit c:\LTU\etc\ltu\database_postgres.dsn

  • For 32bits ODBC driver please download and install the appropriate driver from http://www.postgresql.org/ftp/odbc/versions/msi and set DRIVER to "DRIVER=PostgreSQL Unicode".
[ODBC]
DRIVER=PostgreSQL Unicode(x64)
UID=EDIT::Database_user_name
PWD=EDIT::Database_user_password
PORT=5432
SERVER=EDIT::Database_server_address
DATABASE=EDIT::Database_name
XaOpt=1
LowerCaseIdentifier=0
UseServerSidePrepare=0
ByteaAsLongVarBinary=0
BI=0
TrueIsMinus1=0
DisallowPremature=0
UpdatableCursors=1
LFConversion=1
ExtraSysTablePrefixes=dd_
CancelAsFreeStmt=0
Parse=0
BoolsAsChar=1
UnknownsAsLongVarchar=0
TextAsLongVarchar=1
UseDeclareFetch=0
Ksqo=1
Optimizer=1
CommLog=0
Debug=0
MaxLongVarcharSize=8190
MaxVarcharSize=255
UnknownSizes=0
Socket=4096
Fetch=100
ConnSettings=
ShowSystemTables=0
RowVersioning=0
ShowOidColumn=0
FakeOidIndex=0
Protocol=7.4-1
ReadOnly=0
SSLmode=disable

3.1.2. MSSQL

Edit c:\LTU\etc\ltu\database_mssql.dsn

[ODBC]
DRIVER=SQL Server
UID=EDIT::Database_user_name
PWD=EDIT::Database_user_password
AutoTranslate=No
Network=DBMSSOCN
Address=EDIT::Database_server_address,EDIT::port
DATABASE=EDIT::Database_name
APP=Microsoft Data Access Components
SERVER=EDIT::Database_server_address

3.2. Configuring the System

See Chapter 7, Configuring LTU engine server for details on all parameters that are used in this section

In the file c:\LTU\etc\ltu\global.bat:

  1. Make sure the MAC address is correctly set (parameter LMHOSTID)

  2. Set parameter DISTRIBUTED to 1

  3. Set parameter DSN_FILE to c:\LTU\etc\ltu\database_postgres.dsn for PostGreSQL and c:\LTU\etc\ltu\database_mssql.dsn for MS SQL.

  4. Depending on which modules you require for a specific machine, set the parameters FLXM_RUN, KIMA_RUN, WEKI_RUN and ADMI_RUN to 0 or 1.

  5. If the machine is running a KIMA module, set its unique ID in the parameter KIMA_ID. IDs must have consecutive values between 1 and N where N is the number of KIMAs in your system.

  6. If you use DNA 1,2,3,53 please change USEEXEOPTION to 1.

Run the tool db-manager.bat as described in Chapter 2.

3.3. Start and Test the System

You should manually start the system to verify that it is running correctly. To do this, use the command ltu-start_console.bat on each of the machines which are part of your system (as described in Chapter 4).

To run all modules as services you may install and run them as described in Chapter 5.

4. Initialization of the system

If you want to change the configuration of the system and in particular the KIMA IP address, you should empty the ltu_host table and run db-manager on each host of the system.

You may then restart the whole system.

Chapter 6. Administrating LTU engine server

1. Starting/stopping LTU engine server on Linux

For security purpose, the system user "ltu" owns all LTU engine server files and processes. Still, you must be logged as "root" to start and stop the software.

1.1. Starting LTU engine server

LTU engine server is initially configured for a distributed installation. Additional configuration parameters are explained in Chapter 7, Configuring LTU engine server. They are marked as SKIPPED when ether you run the startup script "ltud".

Start LTU engine server

$ /etc/init.d/ltud start 
License server (23129)                   [ OK ] 
KImage Repository Module (23160)         [ OK ] 
Web KImager Module (23179)               [ OK ] 
ADMInistration Module (23500)            [ OK ] 

Test if the server is running

$ /etc/init.d/ltud status
License server (23129)                   [ OK ] 
KImage Repository Module (23160)         [ OK ] 
Web KImager Module (23179)               [ OK ] 
ADMInistration Module (23500)            [ OK ] 

All activated modules must be running.

To restart LTU engine server, use this command : /etc/init.d/ltud restart

you can also start, get a status or restart only one module: /etc/init.d/ltud [start|status|restart] [weki|kima|admin]

1.2. Stopping LTU engine server

To stop LTU engine server, you may use the same command, except that you use "stop".

$ /etc/init.d/ltud stop 
License Server (23129)                   [ OK ]
KImage Repository Module (23160)         [ OK ] 
Web KImager Module (23487)               [ OK ] 
ADMInistration Module (23500)            [ OK ]

you can also stop only one module: /etc/init.d/ltud stop [weki|kima|admin]

2. Starting/stopping LTU engine server on Windows

The first time you install the software, it is recommended to start the modules in console mode to check that every thing is running correctly. Once, your installation is validated, you may run the software from the Windows service interface.

To start and stop the services, you can use the "Service" program in the Windows Administration Tools. the names of services to launch are LTUADMI, LTUFLXM, LTUKIMA, and LTUWEKI.

You can also use the following scripts:

Open a command prompt window and change to directory "c:\LTU\bin" and type:

ltu-start-services.bat

to start the services and

ltu-stop-services.bat

to stop the services.

3. Activating/Deactivating process monitoring (Linux only)

The process monitoring is done through the MON module. This module checks that the activated modules are running and that the WEKI module answers some basic TCP and HTTP queries.

The MON process is handled by the startup script "/etc/init.d/mond", run it with the arguments start, status and stop to respectively start, check or stop the service.

To completely activate the process monitoring, you need to do the following:

  1. Make sure MON is started

    $ /etc/init.d/mond start 
    Starting mon daemon: [ OK ]
  2. Activate/Deactivate process monitoring using the administration interface

    The MON administration interface is accessible through the administration interface (c.f. Section 4, “Web-based administration interface”).

4. Web-based administration interface

The URL to access the interface is "http://your_server_address:8005/" or "https://your_server_address:8443/" when SSL is activated where "your_server_address" is Internet address of the host where LTU engine server is installed. The default administration ports may be changed in the configuration file /etc/ltu/local.conf.

Initially, LTU engine server is configured with the user "sadmin" (password "sadmin"), to properly setup your system you need to:

  • Log on the system with default user and password.

  • Change the "sadmin" rights and password (for security reasons).

  • Create content administration and search users.

Figure 6.1. Login page

Login page

The Figure 6.1, “Login page” shows the login page; simply type the login name and password ("sadmin" and "sadmin" by default).

The interface contains a menu bar on the top, by moving the mouse on the button "System Admin", you will see the system administration menu. This system administration interface includes the following pages:

  • Rights Management

  • User Activity

  • Process monitoring

  • Data Model Information

  • Configuration

4.1. Rights Management

This interface let you manage the users and their rights. Figure 6.2, “User management page” shows the user management interface. By default, several users are included to the system. All of them having their password equals to their login name. The default users use default profiles that implement the different levels of possible user rights on the system.

Figure 6.2. User management page

User management page

Figure 6.3. Profile management page

Profile management page

To change a profile, just click on the button. Figure 6.3, “Profile management page” show the profile editing windows.

Figure 6.4. Editing a profile: global settings

Editing a profile: global settings

A profile contains two parts: global settings and per image database settings. The global rights part is shown in Figure 6.4, “Editing a profile: global settings”.

Figure 6.5, “Editing a profile: rights on the "Investigation" image database” shows the rights on the "Investigation" image database(repository).

Figure 6.5. Editing a profile: rights on the "Investigation" image database

Editing a profile: rights on the "Investigation" image database

4.2. User Activity

This page is not actived by default in the system administration menu. To activate it, edit and modify the xml file "/etc/ltu/admi/databases/[your_repository_name]/[your_repository_name]_config.xml" on Linux or "[c:\LTU]\etc\ltu\admi\databases\[your_repository_name]\[your_repository_name]_config.xml" on Windows by setting the attribute "repositoryRole" to "disableProject".

All users' main actions are stored in the database; these pages allow the administrator to monitor the users' activity on LTU engine server. The two ways of viewing this activity are:

  • Reports. This tab gives a summary of user activity over a period of time.

  • Logs. This tab allows the analysis of each user action.

Figure 6.6. User activity report

User activity report

The figure above shows the user activity report, the 2 areas in this page are:

  • The search form. This search form is empty when you first go this page. To get all reports enter "%" in the user text entry and press the "Search" button.

  • The search result table. This table contains the user activity summary corresponding to the search criteria entered in the search form.

For each user activity report, you may press the button to get a detailed summary of this user activity.

Figure 6.7. User activity logs

User activity logs

The figure above shows the user activity logs, the 2 areas in this page are:

  • The search form. This search form is empty when you first go this page. To get all logs enter "%" in the user text entry and press the "Search" button.

  • The search result table. This table contains the user activity logs corresponding to the search criteria entered in the search form.

Note

The export feature applies to all the logs and not only the current page or the selection. In the example above, the exported document will contain 81 logs.

4.3. Monitoring LTU engine server processes (Linux only)

This interface is based upon the "mon" open source software. If "mon" is not started, the web interface will give message saying it fails to connect to the monitoring process.

Figure 6.8. Process monitoring summary page

Process monitoring summary page

This intuitive interface enables you to see the activity of service and to trace back any failure.

Figure 6.9. Process monitoring activation page

Process monitoring activation page

Figure 6.9, “Process monitoring activation page” shows you the activation page.

4.4. Configuration parameters

This page displays the current main configuration parameters:

Figure 6.10. System configuration: Global.conf file

System configuration: Global.conf file

LTU engine server 5.0 stores now global parameters in a database table and comes with the possibility to configure the backend configuration from LTU engine server User Interface.

In the case of a distributed installation, backend configuration is centralized and relevant parameters are shared by all servers.

Figure 6.11. System configuration: Ltu_global_conf table

System configuration: Ltu_global_conf table

If you want to modify an existing parameter of Ltu_gobal_conftable, change parameter values here:

5. Administration tools

Many administration programs are delivered with the product. Some of them are documented here.

5.1. Database manager

The tool "db-manager" can be used to:

  • Install the schema in an empty database (db-manager is called during the installation process)

  • Install/uninstall one or more repositories

  • Activate/Desactivate one or more repositories

To run the database manager on Linux:

$ /home/ltu/tools/install/db-manager.sh

To run the database manager on Windows, run the script "[c:/LTU]/tools/install/db-manager.bat"

5.2. Perl programs for managing repository content

Several programs located in '[LTU_HOME]/tools/perl/' may be of great help to manage the repository content and to create new scripts as well. To understand how to use each one, execute it with option '-h'.

The main programs:

Table 6.1. Perl tool programs

addDirectory.plAdd images from a directory in a repository.
emptyEntity.plEmpty an entity table from all its objects.
addRecord.plAdd images and associated records.
createObjectSet.plCreate an object set from an external uploaded image. The object set can then be used for creating a search report.
deleteEntity.plDeletes an instance of an entity. For example, deletes the record with id 467.
deleteObjectSet.plDeletes an object set including external images.
deleteUnusedImages.plDeletes the images that are not related to a record.
extractKeywords.plDownloads the key words from a repository and put them into a XML file.
filterDirectory.plTo be used with Filter plugin: Filters the images of a directory.
getImages.plDownloads the images of a repository and put them into a local directory.
scheduleTask.plLaunches an internal task on an object set.
searchDirectory.plSubmits the images of a directory to a repository and returns the image similarity score.
updateKeywords.plUpdates the key words in the repository coming from a XML file.
searchImage.plSubmits one image to a repository and returns the image similarity score.
statBounceImage.plBounce on N images from the database displaying stats.
compareImages.plCompare 2 images and return score with associated fine comparison analysis files.
statSearchImage.plSearch all directory images displaying stats.

On top of the standard Perl installation packages, optional packages must be installed, see Table 4.3, “Required RPMs on Linux systems”.

5.3. Incident report archive (Linux only)

In case errors occur in LTU engine server, you have the possibility to generate automatically an incident report archive. This archive can be send to LTU support at support@ltutech.com and will speed up the diagnose step and resolution of your problem.

To do that:

$ cd LTU_HOME/tools/incident/ 
$ ./incidentArchive.sh 
Building the incident directory 
Creating the Tar file 
Incident Archive finished 
***** 
***** You can find the incident archive in 
***** Incident_log_YYYY_MMDD_HH_MM_SS.tgz 
***** Please send it to the LTU support 
*****

You can also use the diagnostic tool accessible from LTU engine server User Interface for generating the incident report archive.

The menu System > Diagnostic Tool gives you access to a page on which you can select which log and configuration files you want to download from LTU engine server for diagnostic and troubleshooting purposes. Click on Diagnostic downloads one archive file with the requested files.

6. The task service

LTU engine server includes a task management module that enables to run administration tasks within the backend.

The task service configuration file is "[c:/LTU]/etc/ltu/weki-scheduler.xml". This file declares how and when tasks must be ran.

<?xml version="1.0" encoding="UTF-8"?> 
<task-scheduler xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:noNamespaceSchemaLocation="xml-backingstore.xsd"> 
  <!-- Remove this comments to activate AutoSync and be sure to edit the configuration file
  <task name="Auto Synchronization">
    <class>com.ltu.taskservice.impl.FileSystemSynchro</class>
    <style>cron</style>
    <cron-expression>0 55 01 * * ?</cron-expression>
    <parameters>
      <key type="string" name="autosync.file.settings" value="/etc/ltu/autosync.properties"/> 
    </parameters>
  </task>
-->
  <task name="Log maintenance">
    <class>com.ltu.components.taskservice.impl.LogMaintenance</class>
    <style>cron</style>
    <cron-expression>0 55 01 * * ?</cron-expression>
    <parameters>
      <key type="string" name="log.life_expectancy" value="365"/>
    </parameters>
  </task>
</task-scheduler>

By default, two tasks are included in LTU engine server 5.0:

  • The AutoSync task that will synchronize an image database with the content of a directory. Every new image added in the directory (or in a subdirectory) will be added in the image database. Every image remove from the directory will be removed from the image database. This task does not handle image meta info and is intended to clients who are using only LTU engine server backend. This task is disabled by default. The configuration file "/etc/ltu/autosync.properties" contains the parameter for this task.

  • The log maintenance task that is executed every night to clean the database for user log entries older than a certain amount of days (1 year by default).

Cron Expression

For those unfamiliar with "cron", this means being able to create a firing schedule such as: "At 8:00am every Monday through Friday" or "At 1:30am every last Friday of the month". A "Cron-Expression" is a string comprised of 6 or 7 fields separated by white space. The 6 mandatory and 1 optional fields are as follows:

Table 6.2. Cron-expression parameters

Field NameAllowed ValuesAllowed Special Characters
Seconds0-59, - * /
Minutes0-59, - * /
Hours0-23, - * /
Day-of-month1-31, - * / ? L W C
Month1-12 or JAN-DEC, - * /
Day-of-week1-7 or SUN-SAT, - * / ? L C #
Year (Optional)empty, 1970-2099, - * /

The '*' character is used to specify all values. For example, "*" in the minute field means "every minute".

The '?' character is allowed for the day-of-month and day-of-week fields. It is used to specify 'no specific value'. This is useful when you need to specify something in one of the two fields, but not the other. See the examples below for clarification.

The '-' character is used to specify ranges For example "10-12" in the hour field means "the hours 10, 11 and 12".

The ',' character is used to specify additional values. For example "MON,WED,FRI" in the day-of-week field means "the days Monday, Wednesday, and Friday".

The '/' character is used to specify increments. For example "0/15" in the seconds field means "the seconds 0, 15, 30, and 45". And "5/15" in the seconds field means "the seconds 5, 20, 35, and 50". You can also specify '/' after the '*' character - in this case '*' is equivalent to having '0' before the '/'.

The 'L' character is allowed for the day-of-month and day-of-week fields. This character is short-hand for "last", but it has different meaning in each of the two fields. For example, the value "L" in the day-of-month field means "the last day of the month" - day 31 for January, day 28 for February on non-leap years. If used in the day-of-week field by itself, it simply means "7" or "SAT". But if used in the day-of-week field after another value, it means "the last xxx day of the month" - for example "6L" means "the last Friday of the month". When using the 'L' option, it is important not to specify lists, or ranges of values, as you'll get confusing results.

The 'W' character is allowed for the day-of-month field. This character is used to specify the weekday (Monday-Friday) nearest the given day. As an example, if you were to specify "15W" as the value for the day-of-month field, the meaning is: "the nearest weekday to the 15th of the month". So if the 15th is a Saturday, the trigger will fire on Friday the 14th. If the 15th is a Sunday, the trigger will fire on Monday the 16th. If the 15th is a Tuesday, then it will fire on Tuesday the 15th. However if you specify "1W" as the value for day-of-month, and the 1st is a Saturday, the trigger will fire on Monday the 3rd, as it will not 'jump' over the boundary of a month's days. The 'W' character can only be specified when the day-of-month is a single day, not a range or list of days.

The 'L' and 'W' characters can also be combined for the day-of-month expression to yield 'LW', which translates to "last weekday of the month".

The '#' character is allowed for the day-of-week field. This character is used to specify "the nth" XXX day of the month. For example, the value of "6#3" in the day-of-week field means the third Friday of the month (day 6 = Friday and "#3" = the 3rd one in the month). Other examples: "2#1" = the first Monday of the month and "4#5" = the fifth Wednesday of the month. Note that if you specify "#5" and there is not 5 of the given day-of-week in the month, then no firing will occur that month.

The 'C' character is allowed for the day-of-month and day-of-week fields. This character is short-hand for "calendar". This means values are calculated against the associated calendar, if any. If no calendar is associated, then it is equivalent to having an all-inclusive calendar. A value of "5C" in the day-of-month field means "the first day included by the calendar on or after the 5th". A value of "1C" in the day-of-week field means "the first day included by the calendar on or after Sunday".

The legal characters and the names of months and days of the week are not case sensitive.

Here are some full examples:

Table 6.3. Cron-expression examples

ExpressionMeaning
"0 0 12 * * ?"Fire at 12pm (noon) every day
"0 15 10 ? * *"Fire at 10:15am every day
"0 15 10 * * ?"Fire at 10:15am every day
"0 15 10 * * ? *"Fire at 10:15am every day
"0 15 10 * * ? 2005"Fire at 10:15am every day during the year 2005
"0 * 14 * * ?"Fire every minute starting at 2pm and ending at 2:59pm, every day
"0 0/5 14 * * ?"Fire every 5 minutes starting at 2pm and ending at 2:55pm, every day
"0 0/5 14,18 * * ?"Fire every 5 minutes starting at 2pm and ending at 2:55pm, AND fire every 5 minutes starting at 6pm and ending at 6:55pm, every day
"0 0-5 14 * * ?"Fire every minute starting at 2pm and ending at 2:05pm, every day
"0 10,44 14 ? 3 WED"Fire at 2:10pm and at 2:44pm every Wednesday in the month of March.
"0 15 10 ? * MON-FRI"Fire at 10:15am every Monday, Tuesday, Wednesday, Thursday and Friday
"0 15 10 15 * ?"Fire at 10:15am on the 15th day of every month
"0 15 10 L * ?"Fire at 10:15am on the last day of every month
"0 15 10 ? * 6L"Fire at 10:15am on the last Friday of every month
"0 15 10 ? * 6L 2002-2005"Fire at 10:15am on every last Friday of every month during the years 2002, 2003, 2004 and 2005
"0 15 10 ? * 6#3"Fire at 10:15am on the third Friday of every month

Pay attention to the effects of '?' and '*' in the day-of-week and day-of-month fields!

Notes:

  • Support for the features described for the 'C' character is not complete.

  • Support for specifying both a day-of-week and a day-of-month value is not complete (you'll need to use the '?' character in on of these fields).

  • Be careful when setting fire times between mid-night and 1:00 AM - "daylight savings" can cause a skip or a repeat depending on whether the time moves back or jumps forward.

Chapter 7. Configuring LTU engine server

1. System configuration files

The configuration of LTU engine server is done through configuration files located in "[c:/LTU]/etc/ltu" and through the database table ltu_global_conf.

In this chapter all configuration options are detailed.

This section lists standard configuration parameters, as well as advanced configuration parameters that should not be edited unless under the supervision of an LTU support and consulting team (support@ltutech.com).

All variables are listed as in table below

Table 7.1. Parameters notation

VariableDefaultValue(s)Comments
Variable namedefault valuevalue type or possible valuesSome remarks on the use of this parameter

The configuration files are shell files. Two rules must be followed:

  • All comment must be on a line starting with "#".

  • Variable must be assigned with no spaces between the variable name, the sign "=" and the value. For example, "SERVER_NAME=my_server" is good and "SERVER_NAME= my_server" will produce an error.

The configuration files are:

Table 7.2. Main configuration files

install.conf (Linux only)Local installation directory (not to be edited)
global.conf (Linux only) or global.bat (windows only)Configuration file common to all hosts in the LTU engine server system (this file should be the same on each host).
local.conf (Linux only)Local configuration
database.conf (Linux only)The database connection properties (not to be edited)

The database table which stores global configuration variables is ltu_global_conf.

1.1. Install.conf (Linux only)

Table 7.3. Install.conf parameters

VariableDefaultValue(s)Comments
LTU_HOME/home/ltuDirectoryInstallation directory

1.2. Global.conf or global.bat

1.2.1. Global installation parameters

By default, images (thumbnails and originals) and binary are stored in "[c:/LTU]/var/ltu/repXXX/" (default repository settings), signatures are stored in database.

The content administrator may choose to store them directly in the database (PostGreSQL or MSSQL).

Table 7.4. Global.conf or global.bat installation parameters

VariableDefaultValue(s)Comments
HTTP_ADDR (Linux only)NoneNone server name or IP addressPut the WEKI server address for standard use.
DISTRIBUTEDyes on Linux, 1 on Windowsyes or no on Linux 0 or 1 on WindowsTells the system to work distributed or not
KIMA_64 (Windows only)10 or 11 to run 64bits KIMA server (only on 64bits host), 0 to run 32bits KIMA server (on 32bits or 64bits host)

1.2.2. Database parameters

Table 7.5. Global.conf or global.bat database parameters

VariableDefaultValue(s)Comments
DATABASEPOSTGRESQLPOSTGRESQL or MSSQL(Windows only)Database used
DATABASE_ADDRNoneServer name or IP addressHost name or IP address of the database server
DATABASE_NAMENoneDatabase identifierName of the database
DATABASE_PORT5432 (PostGreSql default port)Port numberDatabase port (5432 is the default PostGreSql one, 1433 is the default MSSQL port )
DATABASE_LOGINNAMENoneUser loginDatabase user login
DATABASE_LOGINPWDNoneUser passwordDatabase user password

1.3. Local.conf (Linux only)

The configuration file "local.conf" contains the parameters specific to each host of the LTU engine server installation

Note

These parameters exist and are also configured with the same values in global.bat (Windows).

.

1.3.1. Local installation parameters

The variable "*_RUN" are used to activate or not each module on each host. When modules are activated they are started by the "ltud" command, otherwise the module is ignored and the status "skipped" is always displayed.

Table 7.6. local.conf installation parameters

VariableDefaultValue(s)Comments
WEKI_RUN10 or 1WEKI module activation
KIMA_RUN10 or 1KIMA module activation
FLXM_RUN10 or 1License server module activation
ADMI_RUN10 or 1Administration module activation
MON_RUN (Linux only)10 or 1Enable process monitoring

1.3.2. KIMA local parameters

The variable KIMA_ID identifies each KIMA module and must be unique among all KIMA modules of the system. Each module loads the categories according to this id. Two KIMA modules having the same id would load the same images in memory leading to conflicts.

Table 7.7. local.conf KIMA parameters

VariableDefaultValue(s)Comments
KIMA_PORT8091NumberKIMA listening port
KIMA_ID1NumberThe KIMA unique identification number. Must be <= KIMA_NUMBER
KIMA_NUM_THREADS10NumberNumber of thread for kima. Must be equal to 2 times the number of core allocated to the kima(0: unlimited)

1.3.3. WEKI local parameters

Table 7.8. local.conf WEKI parameters

VariableDefaultValue(s)Comments
WEKI_MEM_SIZE1500 on Linux 512 on WindowsNumberMaximum memory size allowed for the java virtual machine in MBytes.
WEKI_PORT8090Port numberListening port for LTU engine server request
WEKI_DATABASE_POOLSIZE (Linux) DATABASE_POOLSIZE (Windows)5 on Linux 4 on WindowsNumberDatabase connection pool size (do not edit).
WEKI_NUM_THREADS10NumberNumber of threads allocated to this weki (i.e. number of parallel processing). It must be consistent with your license and with the configuration of the other ltu modules (the kima server(s) and/or the other weki server(s)).

1.3.4. ADMI local parameters

Table 7.9. local.conf ADMI parameters

VariableDefaultValue(s)Comments
USE_SYSTEM_HTTPD (Linux only)1Number 
USE_SSL (Linux only)00 or 1Secure web based administration with SSL
ADMI_APACHE_PORT (Linux only)8005Port numberAdministration listening port
ADMI_APACHE_SSL_PORT (Linux only)8443Port numberAdministration listening port when using SSL.

1.3.5. MON local parameters

The process monitor program "mon" is by default only watching processes to determine whether they are running or not. Additionally, it can also monitor connections to the WEKI module to verify regularly if the HTTP module is up and ready.

Table 7.10. Install.conf MON parameters

VariableDefaultValue(s)Comments
MON_WEKI_SERVICE_TCP (Linux only)10 or 1Monitor the WEKI module attempting TCP connection regularly.
MON_WEKI_SERVICE_HTTP (Linux only)00 or 1Monitor the WEKI module attempting HTTP connection regularly.
ADMINISTRATOR_MAIL (Linux only)   

1.3.6. Other parameters

Table 7.11. Other parameters

VariableDefaultValue(s)Comments
OMP_NUM_THREADS10 or 1Change it only on CentOs 5 64bits and Debian 6 64bits. Comment the line in local.conf to index images with max core number export


1.4. Database.conf (Linux only)

This configuration file should not be accessed .

1.5. Global parameters in "ltu_global_conf" table

The parameters in ltu_global_conf table are advanced configuration parameters. After changing any of these values, you must restart the backend. Please for more informations, contact LTU support LTU at support@ltutech.com

Table 7.12. Variables handled by ltu_global_conf table

KeyValueComments
api.version4.0Do not edit
cache.dna.impl Ask support
classification.classif.threshold Keyword suggestion classification threshold
classification.enabled.repositories Repositories on which keyword suggestion is activated
db.version4.7Do not edit
entityservice.impl Ask support
http.max.post.size Maximum image size in byte
http.response.encoding Backend response encoding
image.classifier.impl Ask support
image.codec.impl Ask support
image.encryption.enable Encrypt images in database - yes or no
image.entityservice.impl Ask support
image.max.surface Maximum image surface in pixels
image.min.size Maximum image height or width
image.process.impl Ask support
internal.cache.impl Ask support
keyword.metric.coefficient Do not edit
keyword.process.impl Ask support
keywordpropagation.enabled.repositories Repositories on which keyword suggestion is activated
keywordpropagationservice.impl Ask support
kimage.process.impl Ask support
metainfosservice.impl Ask support
query.order Ask support
query.use.distinct Ask support
repository.split Distribution partitioning string
retrieval.postprocessing Ask support
retrieval.process.impl Ask support
subsetservice.impl Ask support
thumbnail.size Thumbnail size
xmlbuilder.impl Ask support

Note

You can also view the variables handled in ltu_global_conf table from LTU engine server User Interface from the menu System > Configuration > System.

2. Repository configuration "repNN_ltu_rep_conf"

For a repository NN, an entry in the table repNN_ltu_rep_conf contains the repository parameters which are detailed in Table 7.13, “Repository configuration parameters”.

Table 7.13. Repository configuration parameters

AttributeDefaultValue(s)Comments
signature_home_dir, signature_storage, image_home_dir, image_storage, binary_storage,--See Table 7.14, “Storage location database attributes”.
binary_home_dir/var/ltu/repNN/binary/directoryWhere external images are stored. These images can only be stored on file system in this version. They are images from AutoScan search reports.
default_segment_image00 or 1Whether to segment images or not (0 means do not segment)
default_color_weight50Integer between 0 and 100Default color weight used when not specified in the query (bounceimage or searchimage)
default_prefilter_image00 or 1Whether to prefilter images or not (0 means do not prefilter)
default_nb_results20IntegerDefault number of results when not specified in the query.
nb_results_cached100IntegerNumber of results cached.
default_signature_type-Valid signature numberRefer to Chapter 2, LTU engine server Overview for more information about signature ids.
quantize--Not used anymore.
use_stemming00 or 1Whether or not stem keywords (e.g. girl equals to girls)
use_image_meta_infos00 or 1Whether or not extract EXIF and IPTC information from images.
store_original_image10 or 1Whether or not keep original images
store_thumbnail_image10 or 1Whether or not compute and store thumbnail images.
retrieval_threshold-1.0-1.0 or a float > 0Threshold on retrieval results. Image with a score superior to this threshold will NOT be returned. -1.0 means no threshold.
similar_threshold1.2Float > 0AutoScan similar threshold
clone_threshold1.0Float > 0AutoScan clone threshold
video_match_threshold1.1Float > 0Video matching threshold.
nb_retrieval_threads1IntegerNumber of threads used in the retrieval service
use_combined_keyword_distance0 (1 for signature 2)Integer (0 or 1)use combined keyword distance

Caution

Any change in this table will be take into account at the next service restart.

3. Setting the storage location

LTU engine server 5.0 can store images, signatures and binary on disk or in the database (available with POSTGRESQL and MSSQL). The storage location is configured in the database. Configuration must be done when the image database is empty.

For example, the default image database "photo gallery" is configured to store images and signatures on disk in the directory "[c:/LTU]/var/ltu/repNN/".

For each repository defined in the database, a table named "REPNN_LTU_REP_CONF" contains the main configuration parameters for the repository. In this table, the storage configuration is done with the following attributes defined in Table 7.14, “Storage location database attributes”, the examples come from the default image database "photo bank" which repository id is NN.

Table 7.14. Storage location database attributes

AttributeDefaultValue(s)Comments
SIGNATURE_HOME_DIR[c:/LTU]/var/ltu/rep1/signatures/directoryUsed only if SIGNATURE_STORAGE is set to "file".
SIGNATURE_STORAGEfilefile or dbIf set to "db", the signatures are stored in the database.
IMAGE_HOME_DIR[c:/LTU]/var/ltu/rep1/images/directoryUsed only if IMAGE_STORAGE is set to "file".
IMAGE_STORAGEfilefile or dbIf set to "db", the images are stored in the database.
BINARY_HOME_DIR[c:/LTU]/var/ltu/rep1/images/directoryUsed only if BINARY_STORAGE is set to "file".
BINARY_STORAGEfilefile or dbIf set to "db", the binary are stored in the database

4. Graphical user interface configuration

All the interface configuration files are located in the directory "[c:/LTU]/etc/ltu/admi/".

The main properties that may be configured from a system administrator point of view are in admi_config.xml:

  <!-- 
       Number of queries displayed in the user search history. This is not linked with the real history size, 
       defined in the backend by the LogsMaintenance.
  -->
  <PROPERTY name="search_history_size" type="int" value="20"/>
[...]
  <!-- Number of objects displayed in a print project window -->
  <PROPERTY name="print_project_display_number" type="int" value="500"/>
[...]
  <!-- For mass-edition, mass-deletion or mass-copy, limit the selection size to avoid time limit in php. -->
  <PROPERTY name="default_selection_limit" type="int" value="2000"/>
[...]
  <!-- Number of seconds of inactivity before the session expire -->
  <PROPERTY name="session_expiration_time" type="int" value="30"/>

5. Using SSL (Linux only)

It is possible to secure the access to the web-based interface using SSL. LTU engine server is delivered with a default SSL certificate that should be changed for maximum security. In particular, when you access the LTU engine server server through, your browser should display a warning message (see Figure 7.1, “Internet Explorer warning.”) saying that:

  • The certificate issuer is unknown to your browser.

  • The server name is not the registered name in the certificate.

Figure 7.1. Internet Explorer warning.

Internet Explorer warning.

The user can ignore this warning and continue. However, you may decide to handle SSL certificates yourself.

Note

LTU engine server external user tools "Image Enrollment", "Image-Cropper" and "AutoScan" supports SSL with a server certificate only. It is not possible currently to configure SSL to require client certificates with these tools.

6. Changing service listening ports on a standalone application

In this section, we explain how to change LTU engine server WEKI and ADMI services' listening ports.

6.1. ADMI listening port (Linux only)

You only need to change the appropriate parameter in local.conf: ADMI_APACHE_PORT or ADMI_APACHE_SSL_PORT if running the interface in secure mode. Then you need to restart the service ADMI.

6.2. WEKI listening port

First change the WEKI listening port in local.conf (Linux only): parameter WEKI_PORT. Then you need to change references to the WEKI service in the admi configuration:

  • In each [c:/LTU]/etc/ltu/admi/databases/rep_name/repXXX_config.xml, set the "host" property of the SERVER element to the address of the WEKI server :

    <SERVER host "" port="8090" apiVersion="4.0" logLevel="-1" retry="0" timeout="0"/>
  • In the directory [c:/LTU]/etc/ltu/admi/httpd/, change the apache configuration file httpd_1.3.conf (Red Hat ES 2.1 only), httpd_2.0.conf or httpd_2.2.conf, change the proxy setting of Apache to redirect all LTU engine server API queries on the WEKI :

    ProxyPass /iseeker/ http://localhost:8090/iseeker/
    ProxyPass /ifilter/ http://localhost:8090/ifilter/
    ProxyPass /common/ http://localhost:8090/common/

Chapter 8. LTU engine server files and processes

In this chapter, we describe how the software runs on the system: the files and directories, the processes, the port opened...

The notation "[HOME_LTU]" must be replaced by your installation directory.

On Linux, the installation process adds the user "ltu" to the system. This user owns all LTU engine server files and directories. The system user "ltu" also owns all LTU engine server processes.

1. Software controls (Linux only)

The program to start, stop and status the LTU engine server services is "/etc/init.d/ltud".

Usage: /etc/init.d/ltud [start|stop|restart|status]

The system uses this program to start LTU engine server when the system starts up and to stop it when the system shut down using following symbolic links:

  • Start

    /etc/rc3.d/S98ltud -> /etc/init.d/ltud

    /etc/rc4.d/S98ltud -> /etc/init.d/ltud

    /etc/rc5.d/S98ltud -> /etc/init.d/ltud

  • Stop

    /etc/rc0.d/K10ltud -> /etc/init.d/ltud

    /etc/rc1.d/K10ltud -> /etc/init.d/ltud

    /etc/rc2.d/K10ltud -> /etc/init.d/ltud

    /etc/rc6.d/K10ltud -> /etc/init.d/ltud

The program "/etc/init.d/ltud" runs the programs described below under the user "ltu" (these scripts can be called directly). To understand how to use each one, execute it with option '-h'.

[HOME_LTU]/bin/flxm-startup.sh

[HOME_LTU]/bin/weki-startup.sh

[HOME_LTU]/bin/kima-startup.sh

[HOME_LTU]/bin/admi-startup.sh

2. Software log files

Log files are located in the directory "[c:/LTU]/var/log/ltu/".

Table 8.1. Log file by module

ModuleLog file(s)
ADMIadmi.log, admi_error.log, admi_access.log, admi_user_access.log
KIMAkima.log
WEKIweki.log, weki0.log, weki1.log...
FLXMflxm.log

At each software startup, these log files are saved in the same directory. The date is used as the extension in backup log files name, e.g. "kima.log.20030518-20:17:22".

3. Files and directories

This section describes the content and role of the LTU engine server directories and files.

3.1. Directory [HOME_LTU]

Directories with comments:

  • bin: This directory contains all startup files.

  • Contribs: This directory contains all external packages.

  • Contribs/jakarta-tomcat Tomcat used by the "weki" module.

  • Contribs/jakarta-tomcat/server/lib: contains LTU engine server java libraries.

  • Contribs/perl-libs, Contribs/Smarty, Contribs/j2sdk1.5, Contribs/Torque..., Various external libraries used by modules.

  • Contribs/mon-0.99.2 The process monitoring software(Linux only).

  • Contribs/apache-ant Ant tools for external plugin use.

  • Contribs/apache_php The apache server used for the "admi" module (the web-based interface).

  • tools: This directory contains various software tools to administrate LTU engine server. In particular all the database scripts used for the install and uninstall are located in the subdirectory "database". This directory also includes db-manger tool located in subdirectory "install". This tool is used to manage the repositories.

  • ltuSystems: This directory contains LTU engine server binaries and libraries.

  • ltuSystems/native/ External and internal native libraries.

  • ltuSystems/http_data/htdocs/ Web-based interface files and libraries.

3.2. Directory "[c:/LTU]/etc/ltu"

This directory contains all configuration files.

3.3. Image and signature directory

The directory where the images, binary and signatures are stored is configurable (the parameter is stored in database). The default directory is "[c:/LTU]/var/ltu"

4. Processes

Each LTU engine server module runs one or several processes.

On Linux, the user "ltu", except for the process-monitoring service "mon" described in a later section, owns all processes.

All the processes use libraries and data located in "[HOME_LTU]" and may access configuration files in "[c:/LTU]/etc/ltu".

This section describes the system when configured to use the ORB JacORB (default configuration).

Table 8.2. Running program and port opened by module

ModuleProgram(s)Port(s) openedComments
flxm[HOME_LTU]/bin/win32/lmgrd, [HOME_LTU]/bin/win32/ltutech27000 + Arbitrary portThese programs access the license file in "/etc/ltu".
kima[HOME_LTU]/bin/win32/StandalonekimaServer or [HOME_LTU]/bin/win64/StandalonekimaServer8091 
weki[HOME_LTU]/Contribs/ j2sdk1.5/bin/java8090 + Arbitrary port 
admi[HOME_LTU]/Contribs/ apache_php/bin/httpd or /usr/sbin/httpd8005 or 8443 with SSLUse first program on Red-Hat Enterprise Server 2.1, use the second otherwise

The modules KIMA and WEKI use SOAP to communicate with each other: several ports may be opened at any time by any of these services.

5. The process-monitoring service "mon" (Linux only)

The process monitoring service is run by the system at startup via the script "/etc/init.d/mond". This script dynamically write its own configuration file (based on the local LTU engine server settings) in "[HOME_LTU]/Contribs/mon". The «root» user always owns this service.

This service uses several utility scripts to check the status of LTU engine server services and eventually restart modules if they are not running or if they fail answering basic queries. The table below describes the main characteristics of these programs.

Programs: /usr/bin/perl, /bin/sh

Write access: [HOME_LTU]/Contribs/mon

Read access: [HOME_LTU]/Contribs/perl-libs [HOME_LTU]/run

Execution access: [HOME_LTU]/bin

Network access: IP 127.0.0.1 Port 8090

Chapter 9. Uninstalling LTU engine server

1. Uninstalling on Linux

The uninstall program "ltu-uninstall" is provided with the distribution on the CDROM or in the archive file. Follow the pre-installation tasks of the installation procedure to get access to this program.

Before uninstalling the LTU engine server, make sure that you use the uninstall program provided with the version of the LTU engine server you want to uninstall. (Especially when you need to uninstall LTU engine server for an upgrade, use the uninstall program of the LTU engine server you are about to upgrade).

First, change to the install directory

$ cd /tmp/LTUEngineServerInstall 

or

$ cd /mnt/cdrom 

Then, run ltu-uninstall

$ ./ltu-uninstall

***************************************************************************** 
*                                                                           * 
* LTU-Engine Server uninstall                                               * 
*                                                                           * 
* Copyright (c) 1999-2011 LTU Technologies                                  *              
* All rights reserved                                                       * 
*                                                                           * 
*****************************************************************************

-------------------------------------------------------------------- 
1: Shell libraries 
-------------------------------------------------------------------- 
Loading LTU Technologies main shell library                      [ OK ] 
Loading Linux shell library                                      [ OK ] 
Loading database shell library                                   [ OK ] 
Loading LTU Technologies install shell library                   [ OK ] 

Uninstall log file: /tmp/ltu-uninstall.260308.log 
Refer to this file if you encounter any problem during the uninstall process. 

-------------------------------------------------------------------- 
2: Launch uninstall 
-------------------------------------------------------------------- 
Looking for an installed LTU-Engine Server version               [ OK ]
 --------------------------------------------------------
Uninstall <LTU-Engine-Server> (A)bort/(C)ontinue? [C] C 

The uninstall program indicates the LTU engine server version currently installed on the host and ask you if you really want to uninstall LTU engine server.

Do you want to backup your configuration (Y)es/(N)o? [Y] Y 
Enter a backup directory [/tmp/backup]: /tmp/MyBackUp 
Creating directory /tmp/backup                                   [ OK ] 
Backup main configuration directory /etc/ltu                     [ OK ] 
Configuration backup of LTU-Engine Server                        [ OK ] 

At this point you may choose to backup you configuration files (All files in "/etc/ltu"). If you want to do so, "ltu-uninstall" will ask you to enter the backup directory.

Stopping mon                                                     [ OK ] 
Stopping all modules                                             [ OK ]
 -- Default database storage with the following parameters:
  -   Database user login     : ltu_user
  -   Database user password  : ltu_password

Checking postgresql client binaries                                 [ OK ]
Do you want to uninstall the database (Y)es/(N)o? [N] Y 

Uninstalling the database will erase all your data (A)bort/(C)ontinue? [C] C 

The program indicates the current database parameters and asks you whether you want to uninstall the database. If you choose to uninstall the database, the program asks to confirm.

Caution

If you choose to uninstall the database, all tables and their content will be deleted included all information related to signatures and images you were using with LTU engine server.

Uninstalling the database will erase all your data (A)bort/(C)ontinue? [C] 
Checking needed packages for database module                        [ OK ]
Loading database storage                                            [ OK ]
Restarting database storage                                         [ OK ]
Checking if user ltu_user exists                                    [ OK ]
Drop database ltu_engine                                            [ OK ]
Remove image and signature directory /var/ltu                       [ OK ]
Status of LTU engine server database uninstall:                     [ OK ]
-------------------------------------------------------------------- 
2.1: LTU-Engine Server Software uninstall 
-------------------------------------------------------------------- 

Remove software files (/etc/ltu,/home/ltu,/var/log/ltu,/var/run/ltu) (A)bort/(C)ontinue? [C] C 
Removing directories
 - /etc/ltu                                                     [ OK ]
 - /home/ltu                                                    [ OK ]
 - /var/log/ltu                                                 [ OK ] 
 -  /var/run/ltu                                                    [ OK ]
-------------------------------------------------------------------- 
2.2: Clean-up the machine 
-------------------------------------------------------------------- 
Delete user ltu (A)bort/(C)ontinue? [C] C 
Delete user ltu                                                 [ OK ] 
Unregister LTU-Engine-Server services                           [ OK ] 
Unregister mon service                                          [ OK ] 
Delete ltu service system start script                          [ OK ] 
Delete mon service system start script                          [ OK ] 
-------------------------------------------------------------------- 
Status of LTU-Engine Server Software uninstall:                 [ OK ] 
-------------------------------------------------------------------- 

STATUS: HOST your_hostname UNINSTALL ENDED SUCCESSFULLY

The uninstall program stop all processes and remove all LTU files. You may choose to skip any of the uninstall step by answering "A" to the confirmation questions.

2. Uninstalling on Windows

Before uninstalling the software you should make sure that the LTU services are not running anymore.

Then, open a command prompt window and change to directory c:\LTU\tools\install. Run the following script:

ltu-uninstall-services.bat

This script should uninstall the four Windows services named "LTUADMI (Apache 2)", "LTUFLXM", "LTUKIMA" and "LTUWEKI (Tomcat server 5)".

To completely remove LTU engine server from the machine, you can simply remove the directory "c:/LTU". If you want to keep your data, you should before save "c:/LTU/var/ltu".

Appendix A. Supported Image Formats

These are the image formats supported by LTU engine server. For most of these formats, LTU engine server does not use the extension of the file to recognize the format; it uses special information inside the binary content (i.e. a JPEG image rename with an ".exe" extension will be recognized as an image). For some image formats, LTU engine server needs the file extension; these formats are marked with "*".

Table A.1. Supported Image Format

FormatDescriptionPossible extensions
AVS*AVS Xavs
BMPMicrosoft Windows bitmapbmp, dib, vga, bga, rle, rl4, rl8
DCXZsoft IBM Multi-page Paintbrushdcx
FITSFlexible Image Transport System (not supported in thi release)fits
GIFCompuServe Graphics Interchangegif, giff
ICO*Microsoft iconico
JP2JPEG-2000 JP2 (linux only)jp2
JPEGJoint Photographic Experts Groupjpeg, jpg, jfif
MIFFMagick image file formatmiff
MTV*MTV Raytracingmtv
PCDPhoto CDpcd
PCXZSoft IBM PC Paintbrushpcx, pcc
PICTApple Macintosh QuickDrawpict, pict2
PIX*Alias/Wavefront RLE image formatpix
PNGPortable Network Graphicspng
PSDAdobe Photoshop bitmap filepsd
SUNSun Rasterfileras, rast, sun, sr, scr
SCT*Scitex Continuous Tonesct
RLEUtah Run Length encodingrle
SGIIrix RGBsgi
TGA*Truevision Targatga, vda, icb, targa, vst
TIFFTagged-Image File Formattif, tiff
VICARPDS/VICARvicar, pds
VIFFKhoros Visualizationviff, xv
XCFGIMPxcf
WBMP*Wireless Bitmapwbmp