ModularRepository

Overview

The Repository stores peer data in the form of MeasurementRecords. MeasuremenRecords are either published by Peers based on their actual (peer-wise or other) measurements, or inferred by various network data sources, such as WHOIS or ALTO.

The Repository server is implemented as a network service, operating on a dedicated TCP port. A WEB GUI (HTTP management interface) is also provided, and can be accessed as

http://<repAddress>:<repPort>/Manage

The following screenshot shows the management GUI of the ModularRepository server:

By default, Repository data is stored in a MySQL database, thus requires a MySQLD server set up on the same host. Alternatively, for small installations, this implementation also support an in-memory database backend as well.

Description

ModularRepository is the server-side reference implementation of the NAPA-WINE "Repository" concept.

In the NAPA-WINE architecture, repositories are envisioned as global databases, containing network-wide information to aid peer selection decisions. Repositories are key elements in the design to achieve network awareness, and additionally they are instrumental in solving the P2P bootstrapping problem.

The NAPA-WINE architecture defines three different kind of repositories based on the information they contain:

• P-REP: The peer repository is a global repository storing information about the peers� status and end-to-end measurements performed by the peers� monitoring modules.
• N-REP: Network repositories store network-wide information inferred from P-REP values.
• E-REP: The external repository provides network information not directly available to or obtainable by peers. This repository is used by network operations to inject non-public information to aid the peers to form optimal P2P swarms, and utilize network links efficiently.

ModularRepository implements a P-REP out of the box, and supports the easy creation of plugins implementing N-REP and E-REP functionality. In the distibution, one such plugin, a WHOIS plugin is provided as an example. This plugin retrieves the Autonomous_System Number (ASN) and the Country_Code for each peer which have published measurement records to the Repository. Thus, ModularRepository may implement all three kinds of repositories in a single instance, or can be configured to provide just one of them.

The ModularRepository is implemented as a standalone daemon program, written in Java.

Features:

• Implements the repository protocol described by the document "D4.1 API Definition"
• Built-in support for Peer-REP functionality (measurement database)
• Support for Ext-REP plugins (supplement data for PeerIDs extracted from published MeasurementRecords)
  • Currently, an example WHOIS plugin is implemented, which retrieves and stores the AS Number and County Code (IT, HU, etc.) for all published PeerIDs
  • Based on this, an ALTO plugin could be easily implemented
• Plugin support for Net-REP modules
• Support for persistent and non-persistent database storage of MeasurementRecords
• Support for historic (all records) and latest only storage modes for MeasurementRecords
• AutoPurge: automatically purge records older than n. seconds
• Web-based management interface

The ModularRepository is the reference implementation of the Napa-Wine Repository server interface. The Repository is an optional but strongly recommended piece of the NAPA-WINE P2P infrastructure which enhances adaptability and network- friendliness of the P2P overlay.

Repository Protocol

Precise specification of the Repository protocol can be found in svn://repository.napa-wine.eu/napawine-software/trunk/NAPAWINE-BASELIBS/include/repoclient.h, as well as the related NAPA-WINE deliverables.

License

The program is released under the LGPL license.

Usage

The template script start_repository.sh shows how to start up the repository server. Using a Sun JRE (1.5 and above) is recommended. See the script start_repository.sh for invocation details.

For client-side connection to the Repository, please refer to the NapaLibraries distribution, as it contains an example repository client.

Documentation

The distribution contains standard JavaDoc documentation for developers. An on-line copy can be found here

Running a repository

The repository can work either all in memory or (by default) using a MySQl^? database. In the later case a running instance of MySql^? is required. Please refer to teh documentation of distribution on how to install MySQL^?.

Download

Binary download

A pre-packaged binaries can be downloaded from the Hudson build system at the following URL:

http://repository.napa-wine.eu:9836/hudson/job/ModularRepository/lastSuccessfulBuild/artifact/ModularRepository/

Once downloaded (assuming ModularRepository-2196.tgz as downloaded file):

1) Uncompress the archive

tar xzvf ModularRepository-2196.tgz

2) Change directory

cd ModularRepository

3) Setup the repository properties editing the configuration file:

vim repository.properties

In particular you need to set a dbUser and dbPassword to the username and password to be used to create/access the new DB for the repository (see also configuration section below).

3a) (Optional, required only if using MySQL) Create the initial DB using the MySQLSetup.sh script supplying the adminuser and password (to have the rights to create the new DB)

./MySQLSetup.sh --dbadminuser=<admin_user> --dbadminpwd=<password>

admin_user: MySQL admin user having the rights to craete a new db

password: tha password for the above user

NOTE: To speed up WHOIS lookups, the MySQLSetup.sh script proposes to download a publicly available geoip database. If you want to use it, just download the file to the same directory and run MySQLSetup.sh again.

4) Run the repository:

./start_repository.sh

Source download & Building

The program is available in source form, and can be obtained from the NAPA-WINE SVN repository. For example, for a command-line SVN client, use the following command to check out ModularRepository:

svn co https://repository.napa-wine.eu/svn/napawine-software/trunk/ModularRepository

The package.sh script included in the distribution can be used to build the distribution package (containing the jar archive of the executable). The script will compile the Java sources with a JDK. Using a Sun JDK (1.6 and above) is recommended. It might be necessary to edit scripts package.sh and start_repository.sh if your JDK is installed in a non-standard location (edit the part concerning JAVA_HOME).

Dependencies

The program relies on the following external libraries:

• log4j, the standard Java logging framework
• hsqldb, an in-memory Java Database implementation
• jaminid, a library for creating HTTP-based network servers.

All three libraries are open source and freely distributable. For convenience, the ModularRepository distribution contains all three in binary (.jar) format.

Configuration

Configuration options are specified in the repository.properties file (sample included in the distribution). The properties file (with embedded documentation as comments) has the following format:

# - Management password for the Repository  GUI (default: no password)
#password=yourpassword
# - TCP port for the repository server (default port: 9832)
#port=9834
# - Data persistence (default: use mysql and make data persistent)
#persistentDB=no
# - SwarmVisualizer URL to show on the management GUI
swarmVisualizerUrl=http://repository.napa-wine.eu:9833/
# - database backend selector. We recommend mysql
dbType=mysql
# - database user (schema) and password
#   this is the same as repodbuser in MySQLSetup.sh
dbUser=napa
dbPassword=mySecretDBPassword

# store ALL records and never delete (default:no)
#keepAllRecords=no

#purge records older than n. seconds (default: 300). Valid only if keepAllRecords=no
#recordsMaxAge=900
recordsMaxAge=21600

# - DB file location. Used with the HSQLDB backend only.
#   NOT USED WITH MYSQL
#persistentDBLocation=/opt/svn/ModularRepository

Additionally, the file log4j.xml (sample provided) can be used to configure logging for the repository server.