User Tools

Site Tools


guide:developer

Developer's Guide

Developers wishing to work on OpenTG should become familiar with this document.

Project Objectives

  • Create a stable and fully functioning software release
  • Develop security focused code with an emphasis on data and system security
  • Build a community of users and contributors
  • Innovate beyond traditional BBS design, don't simply clone the derivative

Platform Selection

Before you can begin working with the source code, you must understand the development tool-chain and the architecture of the code.

Operating Systems

Linux is the system used to develop OpenTG. Porting plans for other systems are being considered after the 1.0 release has been made. For simplicity, it is best if a Linux workstation is used, either physical or virtual machine, running a recent distribution. This will provide a consistent developer experience and ensure compatibility across the team members. The main developer, Chris Tusa, uses the NetRunner debian-based distribution.

Future consideration is in-place to use container technology, but this has not been implemented.

Primary Programming Language

The primary programming language for OpenTG is JRuby. JRuby differs from Ruby in that it is written in Java instead of mri-ruby which is written in C. This means that a working and properly versioned Java Runtime Environment or (JRE) must be installed on the developer workstation.

JRuby provides several key advantages, which is why it was initially selected. Here are several reasons why this decision was made:

Portability

Portability to other platforms was a key consideration. Since a JRE can be easily installed on almost all well maintained operating systems, the runtime components can also be dropped in.

Libraries

Ruby has a an incredible ecosystem of libraries that can be used. Many are included in the core and others are available as add-on packages using the 'gem' facility. JRuby also provides a facility to import and use Java Classes and Java JAR file Classes. There are many robust and very mature Java libraries that provide equal or better capabilities than those shipped by Ruby and additionally, many that are not available as a gem.

JDBC

Java provides database connectivity through the JDBC facility. This allows access to almost every major DBMS driver by being able to load a vendor shipped driver. This opens up the project to using databases that may not be available to the C version of Ruby.

Other Programming Languages

WARNING DO NOT introduce other programming languages into the project. While Java and the JVM support additional languages, mixing languages will break code maintainability and introduce complexity. If you cannot accomplish something using Ruby, email the mailing list or submit a Bug for help.

There are places where shell scripts are used. Shell scripting should be limited to functions that are used for wrapping startup of various system services or to prepare the runtime environment. When possible, use the Rake tool for automating tasks, rather than writing shell.

Databases

DOS Based BBS's, particularly those written in TurboPascal such as the original Telegard, leveraged Data Files written to disk. Databases at the time were not readily available for freeware use or able to be embedded into the software, so many programs simply used structured data files. OpenTG uses JDBC to provide access to one or more database engines for storing all of the data for each system.

Embedded vs Remote

Two database operation modes are made available for use.

Embedded mode loads a JAR file that includes a fully working database engine that is included and shipped with the distribution. It requires no setup on the administrator to configure.

Remote mode uses a JDBC driver to connect to a remotely running database server. This can be used for scaled installations or as matter of preference for the administrator.

H2 Database

The embedded database provided with OpenTG is the H2 Database Engine. H2 provides fast, and highly compatible SQL syntax that matches the goals for the project's needs. It was evaluated over several other options and was determined to be the best fit.

PostgreSQL

PostgreSQL has been tested as the remote database. It is not used by the developers, so testing will be limited until QA for releases are made.

What about other database?

See the next section on the ORM; however, supporting many databases systems will increase the workload for our developers and testers. If there is solid justification from the community to support an additional database engine by default, it can be added to the roadmap for consideration.

Sequel ORM

An O.R.M. or “Object Relational Mapper” provides an abstraction layer that easily translates SQL queries to and from the programming language's native Object Oriented classes and provides methods to easily manipulate data and perform queries. OpenTG uses the Sequel gem to provide this facility.

Models

Model classes are created for each database table.

Migrations

Migration files are used to manage and maintain the database schema over the development cycles. This will also simplify upgrades.

JDBC for Sequel

Sequel does not ship with a native H2 database driver, so the JDBC adapter is used. It is also preferred to use the PostgreSQL JDBC driver, rather than the native driver provided by the gem.

MVC Framework

OpenTG utilizes a “Model View Controller” framework for organizing the code and providing a clear separation of program functions. This means that Program Control, Database Calls and Screen Output will never live in the same file. To illustrate this concept within the source code, look at this example from the directory structure:

.
├── class
├── conf
├── db
│   ├── migrations
│   │   ├── 001_create_tables.rb
│   ├── opentg.h2.db
├── lib
│   ├── controllers
│   │   ├── bbslist.rb
│   │   ├── callhistory.rb
│   │   ├── chat.rb
│   │   ├── chatroom.rb
│   │   ├── email.rb
│   │   ├── filearea.rb
│   │   ├── file.rb
│   │   ├── help.rb
│   │   ├── main.rb
│   │   ├── messagearea.rb
│   │   ├── message.rb
│   │   ├── timebank.rb
│   │   └── user.rb
│   ├── dbmodels
│   │   ├── bbslist.rb
│   │   ├── callhistory.rb
│   │   ├── chatroom.rb
│   │   ├── email.rb
│   │   ├── extprogs.rb
│   │   ├── filearea.rb
│   │   ├── file.rb
│   │   ├── group.rb
│   │   ├── migrations
│   │   ├── msgarea.rb
│   │   ├── msg.rb
│   │   ├── network.rb
│   │   ├── session.rb
│   │   └── user.rb
└── tpls
    ├── opentg
    │   ├── bbslist_askbbsname.ftl
    │   ├── bbslist_askbbsurl.ftl
    │   ├── bbslist_askdescription.ftl
    │   ├── bbslist_askhomepage.ftl
    │   ├── bbslist_askstart.ftl
    │   ├── bbslist_asksysop.ftl
    │   ├── bbslist_confirmadd.ftl
    │   ├── bbslist_list_local.ftl
    │   ├── callhistory.ftl
    │   ├── email_list_inbox.ftl
    │   ├── filearea_file_invalid.ftl
    │   ├── filearea_file_metadata.ftl
    │   ├── filearea_jumpto.ftl
    - <output truncated>

Controllers

Located in

bbs/lib/controllers/*

Controllers inherit their properties from a base class of 'Tgcontroller'

Models

Located in

bbs/lib/dbmodels/*

Models inherit their base class from 'Sequel::Model'

IMPORTANT: model files no longer include the schema nor creation of the database table structure. This is handled using Sequel::migration via the Rakefile.

DB Migrations

Migration files are located in:

bbs/db/migrations/*

The naming convention follows a numerical pattern as described in the Sequel documentation.

Views

Views are rendered using the Tgtemplate module. Writing template files are explained in more detail in the Template Guide and not in scope for this document. By placing the program output into separate template files, it provides the developers to easily adjust the look and feel of the program outside of the source code. It also provides a way to more rapidly create spoken language translations, and give administrators the ability to customize or create new themes.

Theme files are stored in

bbs/tpls/<themename>/*

The default themename is 'opentg'. Files are end with a .ftl

Developer Environment & Toolchain

Having a working development environment is the first step in getting started. There are operating system dependencies that may impact the experience. Since OpenTG leverages ANSI colors, it is important that whether you access your Linux instance over a remote SSH session or directly through a Terminal windows on the desktop, the session will need to support ANSI colors. The Bash shell is also recommended to be installed. Others such as dash, ksh, zsh may work; however, several of the scripts written to run the system depend on a working bash executable.

Java & JRuby

JRuby needs Java. Selection of a JRE or JDK is important and versions vary in functionality and may impact the function of the software.

OpenJDK

OpenJDK is readily available with most OS distributions. OpenJDK >= 9 is the preferred runtime.

If you have the openjdk installed in the default location provided by your distribution, confirm it works:

[shell]$ which java
/usr/bin/java

[shell]$ ls -l /usr/bin/java
lrwxrwxrwx 1 root root 22 Mar  9 08:28 /usr/bin/java -> /etc/alternatives/java

[shell]$ java -version
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

In this case, this is OpenJDK 8. But we want version 9. To fix this on debian:

sudo apt-get install openjdk-9-jdk openjdk-9-jre
...
...
[shell]$ java -version
openjdk version "9.0.1"
OpenJDK Runtime Environment (build 9.0.1+11-Debian-1)
OpenJDK 64-Bit Server VM (build 9.0.1+11-Debian-1, mixed mode)

[shell]$ ls -l /usr/bin/java /etc/alternatives/java
lrwxrwxrwx 1 root root 42 Aug 21 11:14 /etc/alternatives/java -> /usr/lib/jvm/java-9-openjdk-amd64/bin/java
lrwxrwxrwx 1 root root 22 Mar  9 08:28 /usr/bin/java -> /etc/alternatives/java

Oracle Java

Oracle make an available JRE for download, but requires you to accept their license. This JDK is sometimes tested and should be compatible, as it is derived from upstream OpenJDK. Once downloaded, you will need to unpack the JRE. It is recommended to place it in the contrib directory of the telegard installation directory:

/opt/telegard/contrib/java/

then symlink the version to the following name:

ln -s /opt/telegard/contrib/java/jdk-10.0.2 /opt/telegard/contrib/jdk 

then update /opt/telegard/runtime.sh and uncomment the JAVA_HOME variable

JAVA_HOME=/opt/telegard/contrib/jdk

This is a recommended approach, you may modify this as you prefer. Again, OpenJDK 9 or later supplied by the OS vendor is preferred and recommended.

Extending Functionality with External Libraries

If new functionality is required that is not provided by the Ruby core or currently added Gems, there are several ways to extend. One is to find a Gem that matches the requirements. Note that we use JRuby, so not all C gems may work. Look for a Java equivalent Gem or pure-ruby gem. Alternatively, you can use a Java library Jar file and wrap it into a class.

Using Ruby Gems

Below is a list of the gems currently required for running OpenTG. (versions may vary depending on the age of this document).

*** LOCAL GEMS ***

bcrypt (3.1.12 java)
cmath (default: 1.0.0)
csv (default: 1.0.0)
fileutils (default: 1.1.0)
ipaddr (default: 1.2.0)
jar-dependencies (default: 0.3.12)
jruby-openssl (default: 0.10.0 java)
jruby-readline (default: 1.2.2 java)
json (default: 2.1.0 java, 1.8.6 java)
psych (default: 3.0.2 java)
rake (12.3.1)
rake-ant (default: 1.0.4)
rdoc (default: 6.0.1, 3.12.2)
rdoc-rake (1.0.1)
scanf (default: 1.0.0)
sequel (5.11.0)
webrick (default: 1.4.2)

Using Java JAR Files

to be written

Using the Debug Shell

OpenTG provides a 'debug' program which wraps the irb (interactive ruby) with a preloaded environment for testing new code and working with the database and other libraries. You must have a working environment and installation of OpenTG available for the shell to work properly.

guide/developer.txt · Last modified: 2018/10/23 21:25 (external edit)