the

project

The functionality mentioned here is what will eventually be available. Not everything will be available in the first release. The plan shows how and when the modules will be made.

All modules interface with the Recipe Executive module. Therefore the interfaces are mostly named after the module the Recipe Executive interfaces with:
Issue Tracker Wrapper
Version Control Wrapper
Filetype Detection
Cross Referencer
Dependency Checker
Automatic Configurer
GUI and Console IDE

Issue Tracker Wrapper

Calling the following an interface between two modules is a stretch. But that's how it fits in the rest of the design, I didn't bother making a separate page for this.

Towards the user two targets are provided, so that he can do:

	:aap report

	:aap tracker

All the recipe commands can be used. A few of them are especially useful for the "report" and "tracker" targets:

• ":do view http://foo.org/tracker": start a browser on the page where issues are to be reported. Possibly a few arguments can be added in the URL. In the future there way also be a way to fill in part of a form. For the time being, the suggestions for text to be entered in certain fields can be written in a file and presented to the user with: ":do view file".
• ":do email {edit} file": send a message to the maintainer or automated tracking system. The file can be filled with relevant information, such as the version number and name of the user. The user will have to edit the message for the actual report.
• ":sys" can be used for any other command, for example ":sys send-pr". Possibly preceded by ":require send-pr" to make sure the program is available.

Version Control Wrapper

In the recipe the "commit" attribute is used on an item that is under version control. It has the form: "XXX://argument". "XXX" specifies the type of version control and is the name of the module. E.g., "cvs://" is used for CVS. The argument is used to specify the location of the version control repository. Its syntax depends on the version control system. The argument can be omitted if the module can figure out the location by itself (e.g., CVS remembers the name of the server after obtaining the initial version).

Each version control module must provide these functions:

XXX_command(url_tail, url_dict, nodelist, action)

Execute a version control command.

XXX_list(url_tail, url_dict, directory, recursive)

List entries currently in the version control system.

url_tail

String: part of the "commit" attribute after XXX://. For: "{commit = cvs://:pserver:user@foo.org/root {path = bar}}" url_tail is: ":pserver:user@foo.org/root". It can be empty, in which case the module must figure out the required parameters itself. Note: %file% must be expanded for each node. You can use repl_file_name() for this (see below).

url_dict

Dictionary of the "commit" attribute and its attributes. For: {commit = cvs://asdf/root {path = foobar}} url_dict is: {"name" : "cvs://asdf/root", "path" : "foobar"}. Useful attributes:

path

path to directory with nodes

logentry

message for commit log

tag

name for this version

nodelist

List of Node objects for which the action is to be carried out. The Node class is defined in the Node module.

action

String: Command to be executed. First argument of the ":verscont" command. Predefined are:

fetch

Update local version from repository. No-op for files that are already up-to-date.

commit

Update the repository for local changes. No-op for a file that didn't change. The file is to be added to the repository when necessary. Do checkout/checkin when checkout is required. Don't change locking of the file.

publish

Like commit and leave the file unlocked.

tag

Add a tag to the current version.

checkout

Like fetch and additionally lock for editing when possible.

checkin

Like commit and leave the file unlocked.

unlock

Remove lock on file, don't change file in repository or locally. No-op for files that aren't locked.

add

Add file to repository. File does exist locally.

remove

Remove file from repository. File may still exist locally.

The function can throw a UserError exception when called with wrong arguments. UserError is defined in the Error module.

When there is something wrong that prevents version control to work (e.g., the repository doesn't exist) it should silently returning zero. A-A-P may try another method to obtain the file.

Further arguments for XXX_list() are:

directory

Absolute path name of the directory for which the listing is requested.

recursive

When non-empty or non-zero: recursively list directories.

repl_file_name(name, node) can be used to replace "%file%" with the name of the node. The result is returned. For example, when "name" is "ftp://foo.org/pub/%file%" and "node.name" is "foo/bar.c" the result is "ftp://foo.org/pub/foo/bar.c". However, in the future something more complicated may be done (esp. to handle changing directory).

Filetype Detection

The Filetype module is implemented in Python. It provides these functions:

ft_detect(fname)

Returns the filetype for the file "fname". Both "fname" and the returned filetype are strings. This uses the default rules and any rules added by the following functions. When the filetype could not be detected None is returned.

ft_check_dir(dir [, errmsg)]

Scan directory "dir" for "*.afd" files, which are loaded with ft_read_file(). When "errmsg" is provided and is non-zero, an error message is given for a non-existing directory. Otherwise it is silently skipped.

ft_read_file(fname)

Read file "fname" for detection rules.

ft_add_rules(str, lnum)

Add file type detection rules from "str". See "filetype.txt" for the syntax. "lnum" is the first line number in the file where "str" comes from. Use one when reading a whole file.

The filetype detection module can also be used as a separate program. This is useful for non-Python programs. The usage is:

	Filetype.py [-I ruledir] ...  [-f rulefile] ... filename

The "-I ruledir" argument can be used to specify a directory to load *.afd (Aap Filetype Detection) files from. These add rules for filetype detection. These are the default directories which are always scanned:

• /usr/local/share/aap/afd/
• ~/.aap/afd/

Cross Referencer

The interfaces are:

Database updating as a program.

The Cross Referencer scans specified files and stores the information about isolated symbols and their context in a database file, replacing existing information for each file. The program is called "aref". This program can be invoked from a shell, directly by the user. Or started with a ":system" command in a recipe. The arguments for the program are:

• The "-u" option, indicating the database is to be updated.
• The list of filenames to be scanned.
• A "-t" option, followed by the name of a filetype. This filetype is used for the following filenames, overriding the automatic detection. "-t filetype" can appear mixed in with the filenames. "-t auto" can be used to go back to automatic detection.
• Optionally the argument "-o" followed by name of the database file to be written. When omitted a database file is created in each directory.

Example: "aref -u foo.c -t text README -t auto bar.h -o AAP/ref"

Database updating as a Python module.

This provides the same functionality as the program, but avoids starting a new interpreter. One function is provided:

   	ref_update(file_dictlist [, outfile])

"file_dictlist" is a list of dictionaries. Each file is one list item and provides a "name" and optionally a "filetype" entry. The "name" should be an absolute path name, not depending on the current directory. When "outfile" is specified it is to be used for the database file. Returns an error message if it fails, None if it works.

Symbol lookup as a program.

Produce grep-like output for a query or start an application for a location. This currently isn't used by other parts of A-A-P, thus it is not further specified.

Symbol lookup through a Python module.

Used by the IDE. Could also be used by an editor.
TODO: this is not used by the Recipe Executive, it will be specified later.

• lookup a symbol literally or with a regexp
• lookup with a query, for example to lookup the members of a class
• specify where to search (file, directory, tree, whole system) and/or which database files to use
• when looking up where a symbol is defined or used, return a list of files, optionally with a list of locations inside the file (much slower)
• possibility to pass text directly instead of a file name, to lookup a symbol in a modified file without saving it

To Recipe Executive: fetch a file.

When a file referenced in the database does not exist, the Cross Referencer may use the Recipe Executive to download it. This interface may also be used by an editor or another program that has used the Cross Referencer to obtain the name of a file.
This is implemented as a Python function in the "Main" module:

   	Main.fetch(filename [, argv])

"filename" must be an absolute path name, not depending on the current directory. The optional "argv" argument is a list of command line arguments that the Recipe Executive uses. Most useful is ["-f", "file.aap"] to use recipe "file.aap" instead of the default "main.aap".
The function returns non-zero for success, zero for failure. By default it will search for a "main.aap" recipe in the directory of the file and directories above it. This recipe must contain information for fetching for the operation to work successfully.

To Recipe Executive: start a command to view a symbol.

The Recipe Executive provides the ":do" command to perform actions on different types of files. The program to be used is specified with recipes and A-A-P commands are used to start the program. The Cross referencer uses the equivalent of ":do view" and ":do edit" to view and/or edit a file.
This is implemented as a generic Python function in the "Main" module:

   	Main.execute(string [, argv])

"string" contains the commands to be executed, just like the A-A-P commands that appear in a recipe. Newlines can be used to separate lines. "argv" is an optional list of command line arguments that the Recipe Executive uses.
The function returns an error message if something went wrong (although this cannot always be detected), None otherwise. The Cross Referencer uses a command like this:

   	:do view {line = 123} {byte = 33} foo.bar

The optional attributes to specify the position in the file are:

• line: line offset in the file. First line is one.
• byte: byte offset from the start of the file, or when "line" is specified, from the start of the line. First byte is one.

To Recipe Executive: update database.

When the Cross Referencer is started with an option to update the database when necessary, it invokes the Recipe Executive to do this. The recipe (either default or specified) contains the list of files to be scanned. The Recipe Executive will then in turn invoke the Cross Referencer for the list of files.
This is implemented with the same Main.execute() function as above. The command with which it is invoked is:

   	:update reference

To Filetype Detection module: detect the type of a file.

This is the interface the Filetype Detection module provides.

Dependency Checker

For the time being, no specific items are provided. The checking has to be done with system or Python commands. The output of these commands must result in a file with the detected dependencies. The format of this file is like a dependency: "target : file1 file2". This means that commands used to produce Makefile dependencies should work.

Automatic Configurer

GUI and Console IDE

The IDE interfaces with many other modules, because it forms a framework for modules to work together. Additionally, there are interfaces to external applications, such as editors, debuggers and viewers. For completeness these will be briefly described as well, because we want the interfaces to be consistent.

IDE using the Recipe Executive

store project settings

The settings are stored in a recipe. To be able to read them back a number of rules have to be defined how the settings are stored. These restrict the syntax used, otherwise the IDE doesn't understand the settings.
A number of variables is predefined to have a known meaning. These must be set with literal strings (thus no Python code or $var things). At the moment these are defined:

$SOURCE

files to be compiled into $TARGET

$SOURCE_XYZ

files to be compiled into $TARGET_XYZ

$TARGET

result of building $SOURCE (can only be one file)

$TARGET_XYZ

result of building $SOURCE_XYZ (can only be one file)

$INCLUDE

files used but not compiled

$DOC_SOURCE

files used to generate documentation

$DOC

documentation files, not generated

$REFERENCE

list of relevant reference database files

":attr" commands are used to specify version control, URL for publishing, etc. The argument includes the list of files involved (literal strings, as above). To avoid confusion with other ":attr" commands, a comment above it specifies what it is for:

# *IDE: version control

files under version control and the attributes specifying the type and location of the version control system.

# *IDE: published files

files to be published, and the attributes used for publishing.

The IDE can change anything in a recipe that it understands. The user can edit the recipe to have it do non-standard tasks. How these conflicting goals are achieved will be further defined over time.

The IDE will use skeletons for often used tasks. For example, building C code with autoconf. To be able to read back specific items in a recipe, also when the user has edited other parts, the IDE uses comment markers. For example, to mark the name of the file that stores the configure arguments: "# *IDE: configure arguments = config.arg".

Obtaining the actual list of nodes

The IDE invokes the recipe executive module to obtain the list of nodes used in the recipe and its included and child recipes. The recipe executive provides this function:

   	nodelist, dict = get_nodelist(filename)

"filename" is the name of the recipe file. The result is a tuple of two items. The first is a list of Node objects, which includes all the files mentioned in the recipes as being used as source and/or target and their attributes. Additionally a dictionary is returned that contains the variables defined in the recipe. The value of $SOURCE, $TARGET, etc. can be obtained from this (e.g., dict["SOURCE"] is the value of $SOURCE). The IDE can now present the files mentioned in the predefined variables in a nice way, and the remaining nodes in another way.

For a recipe that was generated by the IDE all the nodes should be in the predefined variables, so that the IDE can change these lists. For manually created recipes the IDE can offer the user the list of files, so that they can be viewed, but removing them from the recipe won't be possible, because the IDE doesn't know how the nodes were defined. Only the parts that look just like they were created with the IDE can be changed from the IDE.

Obtaining the tree of included and child recipes

The IDE should be able to handle a project that is split up in sub-projects. For each sub-project a separate recipe is used. The $SOURCE_XYZ and $TARGET_XYZ variables define the files in sub-project XYZ. Since the ":child filename" commands are easy to parse, the IDE can do this by itself without interfacing with the recipe executive. The IDE is also able to update a recipe to add a new sub-project or delete one.
Included recipes are not used by the IDE. It will not be able to change items in them.

Obtaining the actual list of variants

The IDE offers the user the choice to select one of the variants to build. This simply works by passing "VAR=value" arguments to the recipe executive.
The IDE parses the recipe itself to find the variants, since they are easy to spot. The IDE can also change the variants by offering a dialog to the user and updating the recipe. However, this only works for variants that are easy to understand by the IDE.

Fetch files

The IDE invokes the recipe executive to fetch files that the user wants to view and are missing or the user has ordered to be fetched. This uses the the same Main.execute() function as above. The command with which it is invoked is one of:

   	:fetch filename
	:fetchall

Note that the IDE must write any updates to the recipe before invoking Main.execute().

Version control

The IDE assumes that all the files in the recipe are either in the same version control system or not version controlled. When the IDE adds or edits the location of the version control system, it uses an ":attr" command as mentioned above. A command with the same attributes but a different list of files may appear in child recipes. Other ways to specify the version control will not be understood by the IDE.
The IDE invokes the recipe executive to perform version control commands. This uses the the same Main.execute() function as above. The command with which it is invoked is, for example:

   	:commit filename

Building targets

The IDE invokes the recipe executive to build targets. This uses the the same Main.execute() function as above. The command with which it is invoked is, for example:

   	:update progname

A variant and specific recipe can specified with the "argv" argument. The IDE can read the "AAP/log" file to obtain error messages and make a list of locations out if this.

IDE using the Version Control Wrapper and Personal Version Control

How the IDE uses the Personal Version Control is still TODO.

IDE using the Cross Referencer

IDE invoking applications

1. run and wait for completion. Useful for having the user enter information or filtering a file.
2. run asynchronously. Useful for displaying documentation.
3. start and keep in contact. Useful for an editor that displays locations for error messages

The third method requires a bidirectional connection between the IDE and the application. This is TODO. Sketch of how it could work:

		editor = invoke_editor(args, myhandler)
			"myhandler" handles callbacks from editor (e.g.,
			build a target)
		editor.do("open", fname, byte)
		editor.do("moveto", fname, byte)
		editor.cando("errorlist")
			check if editor can handle an errorlist

The IDE offers a window with a list of locations. A viewer or editor can be started on a location, or instructed to jump to a location. The list of locations is obtained by parsing build output or cross referencer query. The list of locations can also be passed to an editor that can handle it (e.g., Vim).