NAME

moinupdate.py -- put pages to a MoinMoin Wiki from various sources

SYNOPSIS

moinupdate.py source-option [option]... target-url [page]...

moinupdate.py --help

DESCRIPTION

Put pages to a MoinMoin Wiki at target-url by the XML-RPC interface (V2). The pages are taken from the source given by the source-option.

OPTIONS

Source options

These options define the source where to get pages from. Exactly one of them needs to be given.

-d source-directory
--directory=source-directory

Page sources are to be found from source-directory. The files must have UTF-8 names and must have the extension given by the -x/--extension option. The content of the file must also use character set UTF-8.

For this source no merging is done. Instead the files are put to the target Wiki unmodified.

--media-source=media-url

Use a MediaWiki Wiki as the source. media-url is the base URL of the MediaWiki Wiki. For this to work the target Wiki must have the MediaWiki parser plugin.

Changes are merged as expected. For this to work option -U/--user must be given and used consistently. I.e. there should be a special user doing the merges and nothing else. The last change done to a page in the target Wiki by this special user is taken as the last synchronization point.

-s source-url
--source=source-url

Use another MoinMoin Wiki as the source.

This is the scenario for which this is designed:

Master and slave Wiki

The Wiki at target-url is the master Wiki. This is typically a public available Wiki.

The Wiki at source-url is a slave Wiki. The slave Wiki is typically a local Wiki which is available while being offline.

Updating the slave Wiki

The slave Wiki is updated regularly by rsync or similar means. I.e. the the data directory of the slave Wiki is made an exact copy of the data directory of the master Wiki including all log files.

Working with the slave Wiki

After the slave Wiki has been updates one may make changes in the slave Wiki exactly as would have been done in the master Wiki.

Updating the master Wiki

After the master Wiki is available again moinupdate is used to synchronize the master with the slave Wiki. This is done by comparing the recent changes in both Wikis. Since the slave Wiki started out as an exact copy of the master there is a point in time where the changes fork. Changes which have been done in the slave after this point are synchronized to the master by moinupdate.

Merge options

Options telling what to do when source and target versions need to be merged. These options are ignored if the source doesn't permit merges.

-m merge-mode
--merge=merge-mode

Determines the merge mode used for a page which has been changed in source and target Wiki. merge-mode may be one of the following:

ignore

No merge is done and the page is not updated at all. An error message is output for each such page.

clean

A merge of source and target pages is attempted. If there are no overlapping changes the page is updated. If there are overlapping changes an error message is output and the page is not updated.

save

A merge of source and target pages is attempted. If there are no overlapping changes the page is updated. If there are overlapping changes an error message is output, the merge result with the overlaps is written to directory save-directory given by -S/--save-directory and the page is not updated. This is useful to manually correct overlaps in the saved file and update the target Wiki with the corrected file.

The option -S/--save-directory is mandatory for this case.

force

A merge of source and target pages is attempted and the page is always updated with the result. An error message is output if there are overlapping changes. Note that this may introduce merge markup into the page.

Merges in content are done in the sequence they happened in the source.

If pages are renamed in source and target this is considered unresolvable and an error message is output and the page is not renamed in the target.

Defaults to clean.

-M mail-address
--merge-errors-to=mail-address

Give an e-mail address where merge errors are send to in addition to the normal handling. If an updated page is ignored without merging it is sent by mail. If a merge took place with overlapping changes the page is sentwith overlaps. In both cases the error messages are also sent.

This allows for catching errors even for non-supervised operation of the program.

-S save-directory
--save-directory=save-directory

Gives the directory to save files when -m/--merge is save. In this case this option is mandatory.

--append=append

Gives a string which is appended to every page retrieved from the source.

This must be a valid UTF8 string in the correct syntax for the target Wiki. For MoinMoin target Wikis it may contain standard variables which are substituted while saving. In addition \n is replaced by a new line and @SOURCEPAGE@ is replaced by the name of the source page in the source Wiki. Im most cases there should be a trailing newline.

Currently only implemented for --media-source.

--prepend=prepend

Gives a string which is prepended to every page retrieved from the source. Refer to --append for details.

Currently only implemented for --media-source.

Target options

-a auth-type
--auth-type=auth-type

Gives the type of authorization to be used for the target Wiki.

These options exist:

none

Use no authorization.

basic

Use basic authorization.

moin

Use MoinAuth authorization.

Defaults to basic.

-P password
--password=password

Gives the password to use to login with user account on target-url and source-url.

If this is not given no authentication is done and the target Wiki thus must be completely open for XML-RPC putpage access. Please note that it is crucial that for -s/--source the user accessing the Wikis has the same reading rights in both Wikis.

-U account
--user=account

Gives the name of the user account in the target Wiki under which the pages are written. All information such as recent changes is also read with this user account in the target Wiki and for -s/--source also in the source Wiki.

Defaults to the real name (gecos field) of the current user as retrieved from the operating system. For this all whitespace is removed from the real name and capitalization is kept.

If account is an empty string --auth-type=none is implied.

This must be an UTF-8 string if it contains non-ASCII characters.

General options

-n
--dry-run

Do everything except really changing something in the target Wiki.

-p page-dict
--page-dict page-dict

page-dict gives the name of a page in the target Wiki containing a MoinMoin Dict. This Dict maps names of source pages given by page to names of target pages. It is no error if a page is not mapped by the Dict.

page-dict must be in the same format as page. For the format of a MoinMoin Dict please consult the MoinMoin sources :-( .

The idea of that option is that in the target Wiki you can maintain a list of page name mappings for importing pages from another Wiki regularly.

If not given no mapping applies.

If this option is given but no page is given the keys from this Dict are used as page.

This option is currently supported by source option --media-source only.

-v
--verbose

Operate verbose.

-x extension
--extension=extension

Extension part for all files with page content. This is added to page names for generation of a proper file name for source option -d/--directory and to saved files when -m/--merge is save.

Defaults to .moin.

Arguments

target-url

Gives the base URL of the target Wiki where pages should be put to.

page

Gives a page to put to the target Wiki. The page in the Wiki is named exactly page and the name must be valid UTF-8.

May be given any number of times.

If not given all possible pages in that source are considered. For -s/--source this is the whole Wiki. For -d/--directory this is the whole tree of files with the extension given by -x/--extension. For --media-source this is not yet implemented.

PREREQUISITES

Due to limitations in the standard XML-RPC calls some restrictions apply unless the respective Wiki has the remoteMoin XML-RPC plugins installed.

Deletions in the source Wiki

Handled if the target Wiki has the plugins. Otherwise deletions are handled as merge errors.

Transfer of comments

Handled only if both Wikis have the plugins. Otherwise all operations loose the comments.

Renaming of pages

Handled only if both Wikis have the plugins. If source Wiki does not have the plugins renames are not recognized and original pages still exist. If target Wiki does not have the plugins the rename is replaced by a delete / creation pair destroying the page history.

Using standalone server

For synching two MoinMoin Wikis using -s/--source it is crucial that the recent changes are retrieved by the same users (or at least by users with the same rights to read pages in both Wikis). For the standalone server version there is no simple way to authorize an XML-RPC call, however. If you want to use http authorization you need to apply this patch to request.py:

  --- MoinMoin/request.py.orig  2006-06-21 11:27:47.000000000 +0200
  +++ MoinMoin/request.py       2006-10-01 13:38:01.000000000 +0200
  @@ -1646,6 +1646,18 @@

               ##self.debugEnvironment(sa.headers)

  +            # Create an environment for authorization checks -- StefanMerten
  +            self.env = { }
  +            authorization = sa.headers.getheader('authorization')
  +            if authorization:
  +                # Check for HTTP basic authorization
  +                ( auth_type, user_password, ) = authorization.split()
  +                from base64 import decodestring
  +                ( user, password, ) = decodestring(user_password).split(':', 1)
  +                # Simply believe it's true...
  +                self.env['AUTH_TYPE'] = auth_type
  +                self.env['REMOTE_USER'] = user
  +
               RequestBase.__init__(self, properties)

           except Exception, err:

SEE ALSO

remoteMoin contains a couple of XML-RPC plugins for MoinMoin allowing page deletion over XML-RPC and other things. See

        http://www.merten-home.de/FreeSoftware/remoteMoin/

AUTHOR

Stefan Merten <smerten@oekonux.de>

LICENSE

This program is licensed under the terms of the GPL. See

        http://www.gnu.org/licenses/gpl.txt

AVAILABILTY

See

        http://www.merten-home.de/FreeSoftware/moinupdate/

BUGS

Special changes like added attachments are not considered. This applies also to the changes from PageComment2.

Page reverts are not treated special.

It would be great if changes using -d/--directory could merge. Then this could be used to dump pages, modify/rename them in the filesystem and update the original Wiki with the modified version.

A skip option is needed to skip a given number of changes. This is useful if a previous update attempt failed for some reason and is repeated later.