Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
public:gsoc:2018:aaditya [2018/08/08 18:41]
aadityanair created
public:gsoc:2018:aaditya [2018/08/15 23:19] (current)
aadityanair Added link to recorded videos.
Line 16: Line 16:
  
  
-----+
 ===== Project Related Links ===== ===== Project Related Links =====
   * Project Repository on GitHub [[https://​github.com/​AadityaNair/​ProjectNephos|Project Nephos]]   * Project Repository on GitHub [[https://​github.com/​AadityaNair/​ProjectNephos|Project Nephos]]
-  * Project Documentation ​[[public:gsoc:​python_extension_module_technical_documentation_gsoc_17|Blog Reports]] +  * My blogs on the project ​[[https://​aadityanair.ml/​tags#​GSoC|Blog Reports]] 
- +  * Sample Recordings [[https://​drive.google.com/​open?​id=1elg_2owspB3qSFkG8AEuiKcEbLdIUNAX|here]]
-----+
  
 ===== Installation ===== ===== Installation =====
  
-For installing the Python extension module, the user needs to make sure that the user has compiled/​installed all the dependencies for CCExtractor ([[https://​github.com/​CCExtractor/ccextractor#​debianubuntu|mentioned here]]).+  git clone https://​github.com/​AadityaNair/ProjectNephos.git 
 +  pip install ./​ProjectNephos 
 + 
 + 
 +===== Usage Documentation ===== 
 +Below is how you would manually use Nephos to perform actions manually. This requires the config file to be present. 
 +More information on the config file in the *Configuration* section. 
 + 
 +=== Uploading files === 
 + 
 +  nephos upload <​filename>​ 
 + 
 + 
 +=== Searching === 
 + 
 +  nephos search --name <​name>​ --tags <​tag1>​ <​tag2>​ ... --do_and 
 + 
 +Search for files with ''<​name>''​ and/or tags ''<​tag1>​ <​tag2>​ ...''​. 
 +The and/or part will be decided by the ''​do_and''​ parameter. If specified, all parameters (name, tagswill be joined 
 +by an AND i.e it will search for **"<​name>​ AND <​tag1>​ AND <​tag2>​ ..."​** ​  
 +If not, ANDs will be replaced by ORs. 
 + 
 +Atleast one of ''​--name''​ and ''​--tags''​ is required. 
 + 
 +=== Tagging === 
 + 
 +  nephos tag --for_name <​name>​ --add_tags <​tag1>​ <​tag2>​ ... 
 + 
 +This searches for all instances that contain ''<​name>''​ and for each of them, add the provided tags. 
 + 
 +=== Processing === 
 + 
 +  nephos process <​input_file>​ <​output_file>​
  
-After the user has compiled/​installed all the dependencies for CCExtractor,​ for installing Python extension module, the user needs to install two dependencies:​ +Converts ​the input file to output fileThe formats are guessed ​by their extensions.
-  - SWIG [[http://​www.swig.org/​download.html|swig]] +
-  - Python-dev package [[https://​packages.ubuntu.com/​trusty/​python-dev|ubuntu]] +
-    For other operating systems, the user has to install //​python-dev package// as mentioned in the corresponding ​ system'​s documentation. +
-     +
-After the user has compiled/​installed the additional dependencies for Python Extension module, the user can install the extension module ​by +
-  sudo pip install ccextractor +
-              OR +
-  pip install ccextractor --user +
-(the above method has been successfully tested on unbuntu16.04 - Xenial)+
  
-In case of virtual environments, ​the user can install the Python Extension module by +=== Permissions === 
-  pip install ccextractor +Share uploaded videos with people based on the video tags.
-  ​+
  
-After the user has successfully installed the extension module, the user can test the module as per the [[https://​github.com/​CCExtractor/​ccextractor/​blob/​master/​api/​api_testing.py|test script]] which is used for processing a single sample. +  nephos permission add --for_tags <​tag1>​ <​tag2>​ --share_with <​email>​
-However, if the user desires to process all the samples in a particular directory then the user can use this [[https://​github.com/​CCExtractor/​ccextractor/​blob/​master/​api/​recursive_tester.py|test script]].+
  
-NOTE: //In case the installation fails, make sure you have updated setuptools, pip and also installed/​compiled all the dependencies mentioned in this section.// +This command is persistent. This means that all future videos with the tag will also be shared
----- +To avoid this action pass ''​--not_persistent''​ to the command.
-===== Technical Documentation ===== +
-==== Architectural Documentation ====+
  
-The overall documentation for the extension module as to how has the module architecture designed and how the further contributors can continue development has been done in this [[public:​gsoc:​python_extension_module_technical_documentation_gsoc_17|documentation]]. +Note, The tags provided follow ​the OR semantics. i.e. in the above example, every file with the tag //tag1// 
-==== Dependency Documentation ​ ====+**OR** //tag2// will be shared. ​
  
-For the contributors who want to either just install the dependencies for the extension module [[public:​gsoc:​python_extension_module_compilation_documentation_gsoc_17|documentation]] would help them understand the dependency compilation.+To view all permissions,​
  
 +  nephos permission list
  
  
-==== Contributor Documentation ​ ====+More information can be found for each sub-command by using the ''​--help''​ option after the sub-command ​
  
-For contributors who wish to understand how exactly is the CCExtractor Extension module compiled ​and installed on a system, this [[public:​gsoc:​python_extension_module_compilation_documentation_gsoc_17|documentaion]] contains relevant information+==== Automation ==== 
 +For the most part you want to just specify what to record ​and when leave Nephos at itFor that:
  
-----+=== Add channels === 
 +Add channel to specify where to download stuff from
  
-===== Contributions [PRs/​Commits] =====+  nephos channel add --name '​CNN'​ --ip_string '​1.2.3.4:​5678'​
  
-All the **commits made by me to the CCExtractor repository** could be found [[https://​github.com/​CCExtractor/​ccextractor/​commits/​master?​author=Diptanshu8|here]].+Note that the ''​name''​ should ​be unique for each channel.
  
-All the **pull requests by me to the CCExtractor repository** could be found [[https://​github.com/​CCExtractor/​ccextractor/​pulls?​q=is%3Apr+author%3ADiptanshu8+is%3Aclosed|here]].+== To view added channels==
  
-----+  nephos channel list
  
-===== Bugs Tracked ===== 
  
-  * I have worked on solving [[https://​github.com/​CCExtractor/​ccextractor/​issues/​705|issue #705]] and submitted a [[https://​github.com/​CCExtractor/​ccextractor/​pull/​721|PR(merged)]],​ thus solving the issue+=== Add job=== 
-  * I have also worked on [[https://​github.com/​CCExtractor/​ccextractor/​issues/​304|issue #304]]. I analyzed the issue'​s sample and found some errors in the way the sample information was encoded. A detailed discussion is present at our [[https://​rhccgsoc15.slack.com|Slack]] under channel [[https://​rhccgsoc15.slack.com/​messages/​C5QJXUSDU|bug304]].+Specify when to download other post download options.
  
-----+  nephos job add --name <​jobname> ​--channel <​channel>​ --start <​starttime>​ --duration <​length>​ \ 
 +                 ​--upload --convert_to <​format>​ --tag <​tag1>​ <​tag2>​
  
-===== GSOC 2017 Experience =====+Following are mandatory arguments:​\\  
 +''​--name''​ is the name of the job. This should be unique for each job.\\  
 +''​--channel''​ is the name of the associated channel. This channel should have already been added by the ''​channel add''​ subcommand.\\  
 +''​--start''​ is the start time of the job written in the popular cron format. For more info on the format go [[http://​www.nncron.ru/​help/​EN/​working/​cron-format.htm|here]]. This was used as an reference.\\  
 +''​--duration''​ is how long you want to record. This is provided in minutes.\\ ​
  
-The overall GSOC experience was really wonderful and the entire team here at CCExtractor was really helpfulThe project was really challenging and helped me learn great deal of basic as well as advanced concepts for software development.+Rest are optional arguments:​\\  
 +''​--upload''​ instructs nephos to upload ​the file to Google DriveThis will most likely be the default case in the future versions. In such case, this option will be removed.\\  
 +''​--convert_to''​ makes so that the downloaded file is converted to the provided format before being uploaded.\\  
 +''​--tag''​ tags the uploaded file with the provided tags.\\ 
  
-The project included additions in CCExtractor source code as well as additions in Python extension module part to continue ​the processing of captions via PythonThus, the project was really helpful in exploring concepts of C as well as Python. In addition to these, I also learnt a number of technologies such as SWIG, Python C-API, GCCcompiler basics as well as advanced topics, GDB, etc.+Note that ''​--tag''​ is dependent providing ​the ''​--upload''​ optionIf it not provided ''​--tag''​ is a NOOP.
  
-Moreover, this project helped me get much deeper knowledge of Open Source Software Development   +=== TV Listings === 
 +Nephos also has crude API that supports TV listings.
  
-----+  nephos schedule add --name <​program_name> ​--channel <​channel>​ --start <​starttime>​ --duration <​length>​ --tags <​tag1>​ <​tag2>​
  
-====== About me ======+This syntax is pretty much exactly the same as for the ''​job add''​ above. The `tags` are associated with the program. 
 +This allows for a separate syntax to add a job:
  
-I am final year student studying at IIT Kharagpur, India. I would definitely be an active contributor not only for CCExtractor extension module but also the Sample-Platform as well as CCExtractor+  nephos job add --name <​jobname>​ --program_tags <​tag1>​ <​tag2> ​.. \ 
 +               ​--upload --convert_to <​format>​ --tag <​tag1>​ <​tag2>​
  
-//Email address// : [[[email protected].com]]+This will find all programs with **any** of the provided tags and add them as jobs.
  
-//​Blog// ​         : [[https://​diptanshujamgade.wordpress.com]]+=== Initialise Server === 
 +This starts the orchestration server which is responsible for the record -> process -> upload pipeline. 
 +This will also create all the relevant directories and perform OAuth with google drive, if not done already.
  
-//GitHub profile//: [[https://​github.com/​Diptanshu8]]+  nephos init
  
-//​Slack// ​        : [[https://​rhccgsoc15.slack.com/​team/​skrill|skrill]]+Currently, if a job is added after the server is started, it will not be picked up by the serverSo, make sure you 
 +add all the jobs before starting the server. This will be fixed in a later version.
  
-----+===== Contributing and other information ===== 
 +Currently the project lives on the above provided github link.
  
-=== Thank you===+The [[https://​github.com/​AadityaNair/​ProjectNephos/​wiki|wiki]] contains more information about the internals of the project.
  
 +There is still a lot of stuff that can be improved here. Have a look at the [[https://​github.com/​AadityaNair/​ProjectNephos/​issues|issues]] to know what can be done and don't hesitate to create a new one if you find something new.
  • public/gsoc/2018/aaditya.1533753703.txt.gz
  • Last modified: 2018/08/08 18:41
  • by aadityanair