Add NEWS.txt, bump ChangeLog (w32)

[quvi.git] / doc / HowtoContributeCode

Howto: Contribute code
======================


1. Before you get started
-------------------------

    *) quvi exists primarily for the following reasons
        - Adobe Flash poorly is supported on non-mswindows systems
        - It hogs system resources, exposes system to vulnerabilities

    *) We do not include websites to quvi that
        - Allow downloading already
        - Use the RTMP protocol (at this time)

    *) If you are proposing a change to the library API
        - Consider this carefully, we try to keep API changes minimal
        - Adding is easier


1.1. Tips
---------

    Work with the development code

Grab the code from <git://repo.or.cz/quvi.git>. The web interface can be
found at <http://repo.or.cz/w/quvi.git>.

    Work with git

Although not required, this is certainly recommended. The examples below
demonstrate this. Be sure to name and email in your ~/.gitconfig:

    [user]
    name  = your_name
    email = your_email

    Make libquvi more verbose

libquvi currently supports the following environment settings that may
be used to amp up the verbosity of the library. This may be useful if
you find that the library is not picking up your scripts from the path.

Example:
    env QUVI_SHOW_SCANDIR=1 quvi
    env QUVI_SHOW_SCRIPT=1 quvi

The former causes the library to dump the Lua script search dirs to the
stderr. The latter is similar but prints the found Lua scripts (with full
paths to them) to stderr.

You may also want to use the --verbose-libcurl with quvi(1), this helps
debugging the HTTP traffic generated by the libcurl.


2. Scripting
------------

quvi uses Lua scripts to parse the video details. You can find the
current scripts from $prefix/share/quvi directory. Compare buzzhumor.lua
and youtube.lua scripts to see how different these scripts can be.

As far as the website Lua scripts go, they are by far the most
frequently changing part of the project. Although ideally you would work
with the current development source code, you could just as well work
with precompiled quvi binaries.

The scripts are dynamically loaded when you fire up quvi. You can even
drop new scripts in the path -- or remove and fix old ones without ever
recompiling quvi.


2.1. Choose a website
---------------------

Any flash based website will typically do. Remember, we are trying to
circumvent the need to use Adobe Flash altogether.

How difficult this turns out to be, depends on how the website was
designed. You can get a better understanding by looking over some of
the existing scripts and then looking for similar "fingerprints" in your
website's HTML.

If you are willing to walk an extra mile, find yourself a system that can
execute flash object code (e.g. has the Adobe Flash plugin installed)
and capture the HTTP traffic. You may be able to figure out how the URLs
being constructed by analyzing the captured data. Note that this may not
always be even necessary as some websites often simply list the video
download URL within the HTML.

The (download) URL extraction, whether it is simply copied from the HTML or
put together from different chunks of data is the most time consuming
part of writing your script. Video IDs and titles are typically always
available and can be grabbed from the HTML with little effort.


2.2. Dissecting a website script
--------------------------------

We'll use git and the development code in this example. See 2.3. below,
if you choose not to work with the quvi source code but precompiled quvi
binaries instead.

Let's assume you have figured out how to parse the video details (download
URL, video ID, video title), it's time to cook up a script. We'll go
over the steps below.

Grab the development code from the repo.

    git clone git://repo.or.cz/quvi.git

Use an existing script as a template. buzzhumor.lua will do fine. We'll
use 'foo.lua' for our script's file name for brewity. You should choose
a name that matches the website.

    cd quvi ; mkdir tmp ; cd tmp
    cp ../share/lua/website/buzzhumor ../share/lua/website/foo.lua
    ../configure ; make

Open the foo.lua file in your favorite editor, e.g.:

    vim ../share/lua/website/foo.lua

Start by changing the details in the `ident' function, e.g.:

-   t.domain  = 'buzzhumor.com"
+   t.domain  = 'foo.bar'

You can leave t.formats untouched, unless you know that the website
supports more than one video format -- and you know how to get to those.
See youtube.lua, dailymotion.lua and vimeo.lua for examples of this.

Next there is the `parse' function. This is where the magic happens.
This function is responsible for parsing all of the video details (e.g.
title, ID, download URL and host ID).

-   video.host_id = 'buzzhumor'
+   video.host_id = 'foo'

Let's assume that our website "foo" resembles buzzhumor so that all that
we must do is to fetch a video page from the user specified URL and
parse the video details from the HTML.

    local page = quvi.fetch (video.page_url)

The above fetches the page from the URL (video.page_url) and stores the
contents to a local variable `page'. We go over the page HTML and notice
that the video title can be parsed from <title>.

    local _,_,s = page:find('<title>(.-)</title>')
    video.title = s or error ('no match: video title')

We have now parsed the title and stored it to the `video' table. Next,
the video ID. Next, the video ID. This is usually an unique string of
characters. For example:

    local _,_,s = page:find ('vid_id="(.-)"')
    video.id    = s or error ('no match: video id')

This leaves only the video download URL. This could be as simple as an
URL to an flv file on the server. For example:

    local _,_,s = page:find('vid_url="(.-)"')
    video.url   = {s or error ('no match: video url')}

Note that we store the parsed URL to video.url table ({}). If there were
more than one download URL, we could append them to that table. At the
time of writing this, none of the current scripts provide multiple URLs.

All that remains is to return the updated `video' result table that
contains the parsed details.

To test your new script, make sure you set QUVI_BASEDIR, e.g.:

    (assuming we're still in $top_srcdir/tmp directory)
    env QUVI_BASEDIR=../share ./src/quvi TEST_URL

Tweak the script if needed, e.g. parsing fails or some of the regexps do
not return the expected values. Edit and re-run quvi as described above
until you're happy with the script.

It's time to create the patch. Here's one way to do it:

    (still in the $top_srcdir/tmp)
    git add ../share/lua/website/foo.lua
    git commit -am 'add foo support'
    git format-patch -M -1

And submit the patch to the upstream, see
$prefix/doc/quvi/HowtoSubmitPatches.


2.3. Dissecting a website script using precompiled quvi binaries
----------------------------------------------------------------

We'll use git in this example also so get used to it. git uber alles.
We're not going to get into the gory details of writing a script 
(read 2.2. above) but go over how you can work with only the quvi
binaries on your system.

Make sure you have quvi binaries installed to your path.

    mkdir -p foo/lua/website ; cd foo
    cp $prefix/share/quvi/lua/website/buzzhumor.lua lua/website/foo.lua
    git init ; git add . ; git commit -am 'initial commit'
    (
        * Edit lua/website/foo.lua (as we did in 2.2. above)
        * Run "quvi TEST_URL", you can skip setting QUVI_BASEDIR
        * Repeat above two steps until happy with the results
    )
    git commit -am 'add foo support'
    git format-patch -M -1

And submit the patch to the upstream, see
$prefix/doc/quvi/HowtoSubmitPatches.


3. Tests
--------

Consider writing a test for your website script. Or at very least when
you submit your website script, add a test URL that we can work with
when we write a test for your script.

You can find the current tests from $top_srcdir/tests/ directory.
Note that these support-*.pl scripts depend on the test URLs hardcoded
into the quvi program ($top_srcdir/src/quvi.c).


4. Copyright
------------

If you are going to contribute a new file (e.g. a website script),
please add your copyright to it including your full name and an email
address from which we can reach you. We may need to contact you later.

Otherwise we'll apply the default one:
    Copyright (C) (year)  quvi team <http://quvi.sourceforge.net/>


5. Before you submit your website script
----------------------------------------

    * Does your script parse everything correctly?
        - Host ID
        - Video ID
        - Video title
        - Video download URL

    * Does the website support more than one video format?
        - If yes, see if you can add support for them
        - We can, of course, add the support later

    * Does the parsed video title contain extra characters?
        - We want the video title *only*
        - Anything else, e.g. domain name, should be left out
        - If you are unsure, don't hesitate to ask

    * Check the copyright (if you are submitting a new file)
        - See also "4. Copyright" above