aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorFederico Scrinzi <fox91@anche.no>2012-05-01 15:29:40 +0300
committerFederico Scrinzi <fox91@anche.no>2012-05-01 15:29:40 +0300
commit1c53c60eed6ca8027b9bbb873671ae691f6473a3 (patch)
tree2028b40105a7b8bf03f5cfbfadc5ab7746c0a18d /README.rst
parentREADME in rst (diff)
downloadeuscan-1c53c60eed6ca8027b9bbb873671ae691f6473a3.tar.gz
euscan-1c53c60eed6ca8027b9bbb873671ae691f6473a3.tar.bz2
euscan-1c53c60eed6ca8027b9bbb873671ae691f6473a3.zip
Update README.rst
Diffstat (limited to 'README.rst')
-rw-r--r--README.rst221
1 files changed, 111 insertions, 110 deletions
diff --git a/README.rst b/README.rst
index b6a5ead..f0a1cc3 100644
--- a/README.rst
+++ b/README.rst
@@ -2,7 +2,7 @@ What is euscan ?
================
dev-portage/euscan
-------------------
+==================
euscan is available in portage as a dev package (app-portage/euscan-9999).
This tool allows to check if a given package/ebuild has new upstream versions
@@ -17,56 +17,62 @@ to help euscan do its job more efficiently.
euscan heuristics are described in the "How does-it works ?" section.
-### Examples
+Examples
+--------
+::
+
+ $ euscan amatch
+
+ * dev-ruby/amatch-0.2.7 [gentoo]
-$ euscan amatch
+ Ebuild: /home/euscan/local/usr/portage/dev-ruby/amatch/amatch-0.2.7.ebuild
+ Repository: gentoo
+ Homepage: http://flori.github.com/amatch/
+ Description: Approximate Matching Extension for Ruby
- * dev-ruby/amatch-0.2.7 [gentoo]
+ * SRC_URI is 'mirror://rubygems/amatch-0.2.7.gem'
+ * Using: http://rubygems.org/api/v1/versions/amatch.json
-Ebuild: /home/euscan/local/usr/portage/dev-ruby/amatch/amatch-0.2.7.ebuild
-Repository: gentoo
-Homepage: http://flori.github.com/amatch/
-Description: Approximate Matching Extension for Ruby
+ Upstream Version: 0.2.8 http://rubygems.org/gems/amatch-0.2.8.gem
- * SRC_URI is 'mirror://rubygems/amatch-0.2.7.gem'
- * Using: http://rubygems.org/api/v1/versions/amatch.json
+::
-Upstream Version: 0.2.8 http://rubygems.org/gems/amatch-0.2.8.gem
+ $ euscan rsyslog
-$ euscan rsyslog
+ * app-admin/rsyslog-5.8.5 [gentoo]
- * app-admin/rsyslog-5.8.5 [gentoo]
+ Ebuild: /home/euscan/local/usr/portage/app-admin/rsyslog/rsyslog-5.8.5.ebuild
+ Repository: gentoo
+ Homepage: http://www.rsyslog.com/
+ Description: An enhanced multi-threaded syslogd with database support and more.
-Ebuild: /home/euscan/local/usr/portage/app-admin/rsyslog/rsyslog-5.8.5.ebuild
-Repository: gentoo
-Homepage: http://www.rsyslog.com/
-Description: An enhanced multi-threaded syslogd with database support and more.
+ * SRC_URI is 'http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.5.tar.gz'
+ * Scanning: http://www.rsyslog.com/files/download/rsyslog/rsyslog-${PV}.tar.gz
+ * Scanning: http://www.rsyslog.com/files/download/rsyslog
+ * Generating version from 5.8.5
+ * Brute forcing: http://www.rsyslog.com/files/download/rsyslog/rsyslog-${PV}.tar.gz
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.6.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.7.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.8.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.0.tar.gz ... [ ok ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.10.0.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.11.0.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.1.tar.gz ... [ ok ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.2.tar.gz ... [ ok ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.3.tar.gz ... [ ok ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.12.0.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.4.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.5.tar.gz ... [ !! ]
+ * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.6.tar.gz ... [ !! ]
- * SRC_URI is 'http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.5.tar.gz'
- * Scanning: http://www.rsyslog.com/files/download/rsyslog/rsyslog-${PV}.tar.gz
- * Scanning: http://www.rsyslog.com/files/download/rsyslog
- * Generating version from 5.8.5
- * Brute forcing: http://www.rsyslog.com/files/download/rsyslog/rsyslog-${PV}.tar.gz
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.6.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.7.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.8.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.0.tar.gz ... [ ok ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.10.0.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.11.0.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.1.tar.gz ... [ ok ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.2.tar.gz ... [ ok ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.3.tar.gz ... [ ok ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.12.0.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.4.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.5.tar.gz ... [ !! ]
- * Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.6.tar.gz ... [ !! ]
+ Upstream Version: 5.9.1 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.1.tar.gz
+ Upstream Version: 5.9.0 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.0.tar.gz
+ Upstream Version: 5.9.3 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.3.tar.gz
+ Upstream Version: 5.9.2 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.2.tar.gz
-Upstream Version: 5.9.1 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.1.tar.gz
-Upstream Version: 5.9.0 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.0.tar.gz
-Upstream Version: 5.9.3 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.3.tar.gz
-Upstream Version: 5.9.2 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.2.tar.gz
-### Hidden settings
+Hidden settings
+---------------
You can configure some settings using the command line, but the __init__.py
file of the euscan package contains more settings, including blacklists and
@@ -76,7 +82,7 @@ Maybe we should add the ability to use /etc/euscan.conf and
~/.config/euscan/euscan.conf to override these settings.
euscan-www: euscan as a service
--------------------------------
+===============================
euscan-www is a web application that aggregates euscan results. For example
there is an instance of euscan-www that monitors gentoo-x86 + some official
@@ -86,36 +92,40 @@ euscan-www uses django and provides some custom commands to feed the database.
You can use euscan-www on you system tree, or preferably you can use a local
tree to avoid messing with your system.
-### Installation
+Installation
+------------
-Install requirements from PyPI using
+Install requirements from PyPI using::
$ python setup.py develop
Extra dependencies:
-
- * portage python api
- * rrdtool[python]
+* portage python api
+* rrdtool[python]
Like any django web app, just start by editing settings.py and then run
these two commands.
+::
$ python manage.py syncdb
$ python manage.py migrate
Now your instance is ready, you can just run this command to browse it.
If you want to host it publicly you should use a real webserver.
+::
$ python manage.py runserver
-### Creating a local tree (optional)
+Creating a local tree (optional)
+--------------------------------
Create a local tree with all that portage (and layman would need).
There is an example in euscanwww/scripts/local-tree/. See escan-update.sh
to know what env variables you need to run any portage related command in
this local tree.
-### Scanning process
+Scanning process
+----------------
The scanning process is done by euscan-update.sh. You should read carefully
this script, and adapt it to your needs. For example it uses gparallel to
@@ -123,40 +133,36 @@ launch multiple process at a time, and you should adapt that to your number
of cpu and network bandwith.
Once your euscan-update.sh is ok, just run it.
+::
$ sh euscan-update.sh
-### Custom Django management commands
+Custom Django management commands
+---------------------------------
euscan-www povides some new management commands, here is a short description
of these commands. Use "help" or read euscan-update.sh to get more informations.
-#### list-packages
-
-List packages stored in database.
-
-#### scan-portage
+list-packages
+ List packages stored in database.
-Scan the portage tree and store new packages and versions in the database.
+scan-portage
+ Scan the portage tree and store new packages and versions in the database.
-#### scan-metadata.py
+scan-metadata.py
+ Scan metadata and looks for homepage, maintainers and herds.
-Scan metadata and looks for homepage, maintainers and herds.
+scan-upstream
+ Scan upstream package. The prefered way to use this script it to first launch
+ euscan on some packages, store the result of the file, and feed this command with
+ the result.
-#### scan-upstream
+update-counters
+ Update statistics and rrd files.
-Scan upstream package. The prefered way to use this script it to first launch
-euscan on some packages, store the result of the file, and feed this command with
-the result.
-
-#### update-counters
-
-Update statistics and rrd files.
-
-#### regen-rrds
-
-If you deleted your rrd files, this script will use the database to
-regen them.
+regen-rrds
+ If you deleted your rrd files, this script will use the database to
+ regen them.
How does it work ?
==================
@@ -173,9 +179,12 @@ uses a file that describe what url and regexp to use to match packages.
euscan uses SRC_URI and tries to find the current version (or part of this version)
in the resolved SRC_URI and generate a regexp from that.
-For example for app-accessibility/dash-4.10.1, SRC_URI is:
+For example for app-accessibility/dash-4.10.1, SRC_URI is::
+
mirror://gnome/sources/dasher/4.10/dasher-4.10.1.tar.bz2
-euscan will scan pages based on this template:
+
+euscan will scan pages based on this template::
+
http://ftp.gnome.org/pub/gnome/sources/dasher/${0}.${1}/dasher-${PV}.tar.bz2
Then, from that, it will scan the top-most directory that doesn't depend on
@@ -188,57 +197,49 @@ Like when scanning directories, a template of SRC_URI is built. Then euscan
generate next possible version numbers, and tries to download the url generated
from the template and the new version number.
-For example, running euscan on portage/app-accessibility/festival-freebsoft-utils-0.6:
-SRC_URI is 'http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-0.6.tar.gz'
-Template is http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-${PV}.tar.gz
-Generate version from 0.6: 0.7, 0.8, 0.10, ...
-Try new urls: http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-0.7.tar.gz, etc..
+For example, running euscan on portage/app-accessibility/festival-freebsoft-utils-0.6::
+
+ SRC_URI is 'http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-0.6.tar.gz'
+ Template is http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-${PV}.tar.gz
+ Generate version from 0.6: 0.7, 0.8, 0.10, ...
+ Try new urls: http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-0.7.tar.gz, etc..
Blacklists
----------
euscan uses blacklist for multiple purposes.
-### BLACKLIST_VERSIONS
-
-For versions that should not be checked at all. sys-libs/libstdc++-v3-3.4
-is good example because it's a package which version will always be 3.4
-(Compatibility package for running binaries linked against a pre gcc 3.4 libstdc++).
-
-
-### BLACKLIST_PACKAGES
-
-Some packages are dead, but SRC_URI refers to sources that are still being
-updated, for example: sys-kernel/xbox-sources that uses the same sources as
-vanilla-sources but is not updated the same way.
+BLACKLIST_VERSIONS
+ For versions that should not be checked at all. sys-libs/libstdc++-v3-3.4
+ is good example because it's a package which version will always be 3.4
+ (Compatibility package for running binaries linked against a pre gcc 3.4 libstdc++).
-### SCANDIR_BLACKLIST_URLS
+BLACKLIST_PACKAGES
+ Some packages are dead, but SRC_URI refers to sources that are still being
+ updated, for example: sys-kernel/xbox-sources that uses the same sources as
+ vanilla-sources but is not updated the same way.
-For urls that are not browsable. mirror://gentoo/ is a good example: it's
-both stupid to scan it and very long/expensive.
+SCANDIR_BLACKLIST_URLS
+ For urls that are not browsable. mirror://gentoo/ is a good example: it's
+ both stupid to scan it and very long/expensive.
-### BRUTEFORCE_BLACKLIST_PACKAGES and BRUTEFORCE_BLACKLIST_URLS
+BRUTEFORCE_BLACKLIST_PACKAGES and BRUTEFORCE_BLACKLIST_URLS
+ Disable brute force on those packages and urls. Most of the time it's because
+ upstream is broken and will answer HTTP 200 even if the file doesn't exist.
-Disable brute force on those packages and urls. Most of the time it's because
-upstream is broken and will answer HTTP 200 even if the file doesn't exist.
-
-### ROBOTS_TXT_BLACKLIST_DOMAINS
-
-Don't respect robots.txt for these domains (sourcefourge, berlios, github.com).
+ROBOTS_TXT_BLACKLIST_DOMAINS
+ Don't respect robots.txt for these domains (sourcefourge, berlios, github.com).
Site handlers
-------------
-### Pecl/PEAR
-
-A site handler that uses the Pecl/PEAR rest API
-(http://pear.php.net/manual/en/core.rest.php).
-
-### Rubygems
-
-This one uses rubygems's json API
-(http://guides.rubygems.org/rubygems-org-api/)
+Pecl/PEAR
+ A site handler that uses the Pecl/PEAR rest API
+ (http://pear.php.net/manual/en/core.rest.php).
-### Pypy
+Rubygems
+ This one uses rubygems's json API
+ (http://guides.rubygems.org/rubygems-org-api/)
-Uses pypy's XML rpc API.
+PyPI
+ Uses PyPI's XML rpc API. \ No newline at end of file