diff options
Diffstat (limited to 'docs/plugins')
| -rw-r--r-- | docs/plugins/account_plugin.rst | 11 | ||||
| -rw-r--r-- | docs/plugins/addon_plugin.rst | 163 | ||||
| -rw-r--r-- | docs/plugins/base_plugin.rst | 117 | ||||
| -rw-r--r-- | docs/plugins/crypter_plugin.rst | 69 | ||||
| -rw-r--r-- | docs/plugins/hoster_plugin.rst | 57 | ||||
| -rwxr-xr-x | docs/plugins/overview.rst | 33 |
6 files changed, 450 insertions, 0 deletions
diff --git a/docs/plugins/account_plugin.rst b/docs/plugins/account_plugin.rst new file mode 100644 index 000000000..e683f1604 --- /dev/null +++ b/docs/plugins/account_plugin.rst @@ -0,0 +1,11 @@ +.. _account_plugin: + +Account - Premium Access +======================== + +Example +------- + +MultiHoster +----------- + diff --git a/docs/plugins/addon_plugin.rst b/docs/plugins/addon_plugin.rst new file mode 100644 index 000000000..8f1adf39a --- /dev/null +++ b/docs/plugins/addon_plugin.rst @@ -0,0 +1,163 @@ +.. _write_addons: + +Addon - Add new functionality +============================= + +A Hook is a python file which is located at :file:`pyload/plugins/hooks`. +The :class:`HookManager <pyload.HookManager.HookManager>` will load it automatically on startup. Only one instance exists +over the complete lifetime of pyload. Your hook can interact on various events called by the :class:`HookManager <pyload.HookManager.HookManager>`, +do something completely autonomic and has full access to the :class:`Api <pyload.Api.Api>` and every detail of pyLoad. +The UpdateManager, CaptchaTrader, UnRar and many more are implemented as hooks. + +Hook header +----------- + +Your hook needs to subclass :class:`Hook <pyload.plugins.Hook.Hook>` and will inherit all of its methods, so make sure to check it's documentation! + +All hooks should start with something like this: :: + + from pyload.plugins.Hook import Hook + + class YourHook(Hook): + __name__ = "YourHook" + __version__ = "0.1" + __description__ = "Does really cool stuff" + __config__ = [ ("activated" , "bool" , "Activated" , "True" ) ] + __threaded__ = ["downloadFinished"] + __author_name__ = ("Me") + __author_mail__ = ("me@has-no-mail.com") + +All meta-data is defined in the header, you need at least one option at ``__config__`` so the user can toggle your +hook on and off. Don't overwrite the ``init`` method if not necessary, use ``setup`` instead. + +Using the Config +---------------- + +We are taking a closer look at the ``__config__`` parameter. +You can add more config values as desired by adding tuples of the following format to the config list: ``("name", "type", "description", "default value")``. +When everything went right you can access the config values with ``self.getConfig(name)`` and ``self.setConfig(name,value``. + + +Interacting on Events +--------------------- + +The next step is to think about where your Hook action takes place. + +The easiest way is to overwrite specific methods defined by the :class:`Hook <pyload.plugins.Hook.Hook>` base class. +The name is indicating when the function gets called. +See :class:`Hook <pyload.plugins.Hook.Hook>` page for a complete listing. + +You should be aware of the arguments the hooks are called with, whether its a :class:`PyFile <pyload.PyFile.PyFile>` +or :class:`PyPackage <pyload.PyPackage.PyPackage>` you should read the relevant documentation to know how to access it's great power and manipulate them. + +What a basic excerpt would look like: :: + + from pyload.plugins.Hook import Hook + + class YourHook(Hook): + """ + Your Hook code here. + """ + + def activate(self): + print "Yay, the core is ready let's do some work." + + def downloadFinished(self, pyfile): + print "A Download just finished." + +Another important feature to mention can be seen at the ``__threaded__`` parameter. Function names listed will be executed +in a thread, in order to not block the main thread. This should be used for all kinds of long lived processing tasks. + +Another and more flexible and powerful way is to use the event listener. +All hook methods exists as event and very useful additional events are dispatched by the core. For a little overview look +at :class:`HookManager <pyload.HookManager.HookManager>`. Keep in mind that you can define your own events and other people may listen on them. + +For your convenience it's possible to register listeners automatically via the ``event_map`` attribute. +It requires a `dict` that maps event names to function names or a `list` of function names. It's important that all names are strings :: + + from pyload.plugins.Hook import Hook + + class YourHook(Hook): + """ + Your Hook code here. + """ + event_map = {"downloadFinished" : "doSomeWork", + "allDownloadsFnished": "someMethod", + "coreReady": "initialize"} + + def initialize(self): + print "Initialized." + + def doSomeWork(self, pyfile): + print "This is equivalent to the above example." + + def someMethod(self): + print "The underlying event (allDownloadsFinished) for this method is not available through the base class" + +An advantage of the event listener is that you are able to register and remove the listeners at runtime. +Use `self.manager.listenTo("name", function)`, `self.manager.removeEvent("name", function)` and see doc for +:class:`HookManager <pyload.HookManager.HookManager>`. Contrary to ``event_map``, ``function`` has to be a reference +and **not** a `string`. + +We introduced events because it scales better if there is a huge amount of events and hooks. So all future interactions will be exclusively +available as event and not accessible through overwriting hook methods. However you can safely do this, it will not be removed and is easier to implement. + + +Providing + RPC services +---------------------- + +You may have noticed that pyLoad has an :class:`Api <pyload.Api.Api>`, which can be used internal or called by clients via RPC. +So probably clients want to be able to interact with your hook to request it's state or invoke some action. + +Sounds complicated but is very easy to do. Just use the ``Expose`` decorator: :: + + from pyload.plugins.Hook import Hook, Expose + + class YourHook(Hook): + """ + Your Hook code here. + """ + + @Expose + def invoke(self, arg): + print "Invoked with", arg + +Thats all, it's available via the :class:`Api <pyload.Api.Api>` now. If you want to use it read :ref:`access_api`. +Here is a basic example: :: + + #Assuming client is a ThriftClient or Api object + + print client.getServices() + print client.call(ServiceCall("YourHook", "invoke", "an argument")) + +Providing status information +---------------------------- +Your hook can store information in a ``dict`` that can easily be retrievied via the :class:`Api <pyload.Api.Api>`. + +Just store everything in ``self.info``. :: + + from pyload.plugins.Hook import Hook + + class YourHook(Hook): + """ + Your Hook code here. + """ + + def setup(self): + self.info = {"running": False} + + def activate(self): + self.info["running"] = True + +Usable with: :: + + #Assuming client is a ThriftClient or Api object + + print client.getAllInfo() + +Example +------- + Sorry but you won't find an example here ;-) + + Look at :file:`pyload/plugins/hooks` and you will find plenty examples there. diff --git a/docs/plugins/base_plugin.rst b/docs/plugins/base_plugin.rst new file mode 100644 index 000000000..5fa110fe7 --- /dev/null +++ b/docs/plugins/base_plugin.rst @@ -0,0 +1,117 @@ +.. _base_plugin: + +Base Plugin - And here it begins... +=================================== + +A Plugin in pyLoad is a python file located at one of the subfolders in :file:`pyload/plugins/`. +All different plugin types inherit from :class:`Base <pyload.plugins.Base.Base>`, which defines basic methods +and meta data. You should read this section carefully, because it's the base for all plugin development. It +is also a good idea to look at the class diagram [1]_ for all plugin types to get an overview. +At last you should look at several already existing plugin to get a more detailed idea of how +they have to look like and what is possible with them. + +Meta Data +--------- + +All important data which must be known by pyLoad is set using class attributes pre- and suffixed with ``__``. +An overview of acceptable values can be found in :class:`Base <pyload.plugins.Base.Base>` source code. +Unneeded attributes can be left out, except ``__version__``. Nevertheless please fill out most information +as you can, when you want to submit your plugin to the public repository. + +Additionally :class:`Crypter <pyload.plugins.Crypter.Crypter>` and :class:`Hoster <pyload.plugins.Hoster.Hoster>` +needs to have a specific regexp [2]_ ``__pattern__``. This will be matched against input url's and if a suited +plugin is found it is selected to handle the url. + +For localization pyLoad supports gettext [3]_, to mark strings for translation surround them with ``_("...")``. + +You don't need to subclass :class:`Base <pyload.plugins.Base.Base>` directly, but the +intermediate type according to your plugin. As an example we choose a hoster plugin, but the same is true for all +plugin types. + +How a basic hoster plugin header could look like:: + + from pyload.plugin.Hoster import Hoster + + class MyFileHoster(Hoster): + __version__ = "0.1" + __description__ = _("Short description of the plugin") + __long_description = _("""An even longer description + is not needed for hoster plugins, + but an addon plugin should have it, so the users know what it is doing.""") + +In future examples the meta data will be left out, but remember it's required in every plugin! + +Config Entries +-------------- + +Every plugin is allowed to add entries to the configuration. These are defined via ``__config__`` and consist +of a list with tuples in the format of ``(name, type, verbose_name, default_value)`` or +``(name, type, verbose_name, short_description, default_value)``. + +Example from Youtube plugin:: + + class YoutubeCom: + __config__ = [("quality", "sd;hd;fullhd", _("Quality Setting"), "hd"), + ("fmt", "int", _("FMT Number 0-45"), _("Desired FMT number, look them up at wikipedia"), 0), + (".mp4", "bool", _("Allow .mp4"), True)] + + +At runtime the desired config values can be retrieved with ``self.getConfig(name)`` and set with +``self.setConfig(name, value)``. + +Tagging Guidelines +------------------ + +To categorize a plugin, a list of keywords can be assigned via ``__tags__`` attribute. You may add arbitrary +tags as you like, but please look at this table first to choose your tags. With standardised keywords we can generate +a better overview of the plugins and provide some search criteria. + +=============== ================================================================= +Keyword Meaning +=============== ================================================================= +image Anything related to image(hoster) +video Anything related to video(hoster) +captcha A plugin that needs captcha decrypting +interaction A plugin that makes use of interaction with the user +free A hoster without any premium service +premium_only A hoster only usable with account +ip_check A hoster that checks ip, that can be avoided with reconnect +=============== ================================================================= + +Basic Methods +------------- + +All methods can be looked up at :class:`Base <pyload.plugins.Base.Base>`. To note some important ones: + +The pyload core instance is accessible at ``self.core`` attribute +and the :class:`Api <pyload.Api.Api>` at ``self.core.api`` + +With ``self.load(...)`` you can load any url and get the result. This method is only available to Hoster and Crypter. +For other plugins use ``getURL(...)`` or ``getRequest()``. + +Use ``self.store(...)`` and ``self.retrieve(...)`` to store data persistently into the database. + +Make use of ``logInfo, logError, logWarning, logDebug`` for logging purposes. + +Debugging +--------- + +One of the most important aspects in software programming is debugging. It is especially important +for plugins which heavily rely on external input, which is true for all hoster and crypter plugins. +To enable debugging functionality start pyLoad with the ``-d`` option or enable it in the config. + +You should use ``self.logDebug(msg)`` when ever it is reasonable. It is a good pratice to log server output +or the calculation of results and then check in the log if it really is what you are expecting. + +For further debugging you can install ipython [4]_, and set breakpoints with ``self.core.breakpoint()``. +It will open the python debugger [5]_ and pause the plugin thread. +To open a ipython shell in the running programm use ``self.shell()``. +These methods are useful to gain access to the code flow at runtime and check or modify variables. + + +.. rubric:: Footnotes +.. [1] :ref:`plugin_hierarchy` +.. [2] http://docs.python.org/library/re.html +.. [3] http://docs.python.org/library/gettext.html +.. [4] http://ipython.org/ +.. [5] http://docs.python.org/library/pdb.html
\ No newline at end of file diff --git a/docs/plugins/crypter_plugin.rst b/docs/plugins/crypter_plugin.rst new file mode 100644 index 000000000..b10dd27f9 --- /dev/null +++ b/docs/plugins/crypter_plugin.rst @@ -0,0 +1,69 @@ +.. _crypter_plugin: + +Crypter - Extract links from pages +================================== + +We are starting with the simplest plugin, the :class:`Crypter <pyload.plugins.Crypter.Crypter>`. +It's job is to take an url as input and generate a new package or links, for example by filtering the urls or +loading a page and extracting links from the html code. You need to define the ``__pattern__`` to match +target urls and subclass from :class:`Crypter <pyload.plugins.Crypter.Crypter>`. :: + + from pyload.plugin.Crypter import Crypter + + class MyFileCrypter(Crypter): + __pattern__ = r"mycrypter.com/id/([0-9]+)" + + def decryptURL(self, url): + + urls = ["http://get.pyload.org/src", "http://get.pyload.org/debian", "http://get.pyload.org/win"] + return urls + +You have to overwrite at least one of ``.decryptFile``, ``.decryptURL``, ``.decryptURLs``. The first one +is only useful for container files, whereas the last is useful when it's possible to handle a bunch of urls +at once. If in doubt, just overwrite `decryptURL`. + +Generating Packages +------------------- + +When finished with decrypting just return the urls as list and they will be added to the package. You can also +create new Packages if needed by instantiating a :class:`Package` instance, which will look like the following:: + + from pyload.plugin.Crypter import Crypter, Package + + class MyFileCrypter(Crypter): + + def decryptURL(self, url): + + html = self.load(url) + + # .decrypt_from_content is only an example method here and will return a list of urls + urls = self.decrypt_from_content(html) + return Package("my new package", urls) + +And that's basically all you need to know. Just as a little side-note if you want to use decrypter in +your code you can use:: + + plugin = self.core.pluginManager.loadClass("crypter", "NameOfThePlugin") + # Core instance is needed for decrypting + # decrypted will be a list of urls + decrypted = plugin.decrypt(core, urls) + + +SimpleCrypter +------------- + +For simple crypter services there is the :class:`SimpleCrypter <pyload.plugins.internal.SimpleCrypter.SimpleCrypter>` class which handles most of the workflow. Only the regexp +pattern has to be defined. + +Exmaple:: + + from pyload.plugins.internal.SimpleCrypter import SimpleCrypter + + class MyFileCrypter(SimpleCrypter): + +Testing +------- + +Please append a test link at :file:`tests/crypterlinks.txt` followed by `||xy`, where xy is the number of +expected links/packages to extract. +Our testrunner will be able to check your plugin periodical for functionality.
\ No newline at end of file diff --git a/docs/plugins/hoster_plugin.rst b/docs/plugins/hoster_plugin.rst new file mode 100644 index 000000000..55a973463 --- /dev/null +++ b/docs/plugins/hoster_plugin.rst @@ -0,0 +1,57 @@ +.. _hoster_plugin: + +Hoster - Load files to disk +=========================== + +We head to the next important section, the ``process`` method of your plugin. +In fact the ``process`` method is the only functionality your plugin has to provide, but its always a good idea to split up tasks to not produce spaghetti code. +An example ``process`` function could look like this :: + + from pyload.plugin.Hoster import Hoster + + class MyFileHoster(Hoster): + """ + plugin code + """ + + def setup(): + #TODO + + def process(self, pyfile): + html = self.load(pyfile.url) # load the content of the orginal pyfile.url to html + + # parse the name from the site and set attribute in pyfile + pyfile.name = self.myFunctionToParseTheName(html) + parsed_url = self.myFunctionToParseUrl(html) + + # download the file, destination is determined by pyLoad + self.download(parsed_url) + +You need to know about the :class:`PyFile <pyload.PyFile.PyFile>` class, since an instance of it is given as a parameter to every pyfile. +Some tasks your plugin should handle: check if the file is online, get filename, wait if needed, download the file, etc.. + +Common Tasks +---------- + +Some hosters require you to wait a specific time. Just set the time with ``self.setWait(seconds)`` or +``self.setWait(seconds, True)`` if you want pyLoad to perform a reconnect if needed. + +To handle captcha input just use ``self.decryptCaptcha(url, ...)``, it will be send to clients +or handled by :class:`Addon <pyload.plugins.Addon.Addon>` plugins + + +Online status fetching +---------------------- + +SimpleHoster +------------ + + +Testing +------- + + +Examples +-------- + +The best examples are the already existing plugins in :file:`pyload/plugins/`.
\ No newline at end of file diff --git a/docs/plugins/overview.rst b/docs/plugins/overview.rst new file mode 100755 index 000000000..bbea86756 --- /dev/null +++ b/docs/plugins/overview.rst @@ -0,0 +1,33 @@ +.. _overview: + +================ +Extending pyLoad +================ + +.. pull-quote:: + Any sufficiently advanced technology is indistinguishable from magic. + + -- Arthur C. Clarke + + +.. rubric:: Motivation + +pyLoad offers a comfortable and powerful plugin system to make extensions possible. With it you only need to have some +python knowledge and can just start right away writing your own plugins. This document gives you an overview about the +conceptual part. You should not leave out the :doc:`Base <base_plugin>` part, since it contains basic functionality for all plugin types. +A class diagram visualizing the relationship can be found below [1]_ + +.. rubric:: Contents + +.. toctree:: + + base_plugin.rst + crypter_plugin.rst + hoster_plugin.rst + account_plugin.rst + addon_plugin.rst + + +.. rubric:: Footnotes + +.. [1] :ref:`plugin_hierarchy`
\ No newline at end of file |