You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* replace printable for try/except utf-8
* add sleeps for tests
* moving wait_for_prep_information_job
* just leaving the changes on this pr
* Fixing tests
* Reverting changes
* dist: precise
* addressing @josenavas comments
* fix-2212
* Adding branch description to Contributing (#2256)
* Adding branch description
* Adding spaces
* Adding timeline
* upgrading software
* mv r_client to qiita
* from redis import Redis
* rm moi and ipython
* Move qiita_db/private.py -> qiita_ware/private_plugin.py
* _system_call as system_call
* flake8
* rm create_raw_data
* Transferring VAMPS submission to internal job
* rm other unnecessary files
* addressing @josenavas comment
* Fixing merge conflicts
* Adding tests to private plugin
* rm wrapper.py
* Removing import
* Fixing import
* Modifying GUI to use the plugin
* moving qiita pet in .travis.yml
* Transfer submit to vamps (#2265)
* Move qiita_db/private.py -> qiita_ware/private_plugin.py
* Transferring VAMPS submission to internal job
* Fixing import
* some other .travis.yml fixes
* addressing @josenavas comment
* Adding success test
* adding some methods
* flake8
* Transfer copy raw data (#2267)
* Move qiita_db/private.py -> qiita_ware/private_plugin.py
* Transferring VAMPS submission to internal job
* Fixing merge conflicts
* Adding tests to private plugin
* Removing import
* Fixing import
* Modifying GUI to use the plugin
* Adding success test
* change imports
* adding delete_artifact and create_sample_template
* fixing errors and gui
* fix delete error
* ENH: Make jobs list modal
With some recent changes to the position of the navigation bar, the
header of the jobs list is cut from screen. With this patch in place,
the jobs list will be shown as a modal window, so that won't be a
problem anymore.
* Transfer update delete templates (#2274)
* Moving update_sample_template
* Transfer update_sample_template
* Porting update prep template
* Moving delete sample or column
* Removing tests
* Removing dispatchable and its tests
* Updating interface to use the new functionality'
* Adapting the prep template GUI
* Submitting jobs
* Fixing tests
* Removing qiita_ware/context.py
* flake8ing
* Fixing _system_call
* Safeguarding the call to rollback
* Unmasking more errors
* Forcing different connections on different processes
* Moving job completion to internal plugin structure
* Removing unused code
* Forcing the creation of a new transaction on the jobs
* Fixing tests
* forcing the commit
* Fixing all tests
* Addressing @antgonza's comments
* Addressing @antgonza's comment
* Addressing @ElDeveloper's comments
* BUG: Fix job updates
* Fixing the redis DB (#2277)
* Fixing the redis DB
* Addressing @antgonza's comments
* redbiom install
* redbiom to install_requires
* mv moi-ws to qiita_websocket
* init commit
* addressing @wasade comment
* rm webdis.log
* cleaning code for initial review
* install latest redbiom
* fix test
* ENH: Change phrasing of upload text (#2281)
* ENH: Change phrasing of upload text
As per a user's request.
* Fix typo in docs
Fixes#2259
* addressing @wasade comments and adding other tests
* flake8
* addressing @josenavas comments
* fix#2258
* fix#2258
* fix#858 (#2286)
* addressing @ElDeveloper and @josenavas comments
* @ElDeveloper :|
* redbiom now adds per sample studies to analysis
* jobs-list-as-modal
* addresssing @josenavas comments
* rm () from update_processing_job_data
* add prints to review errors
* rm prints
* Fix 2190 (#2292)
* Adding patch to add the 'name' parameter to all validate commands
* Adding the parameter 'name' automatically and adding a test for it
* Removing extra blank line
* Adding dflt value to artifact name
* Edited the wrong file
* Adding user defined name at creation time
* Fixing test
* add is_from_analysis to artifact_handlers (#2293)
* add is_from_analysis to artifact_handlers
* fix test_post_metadata
* fix error
* WIP: rm sudo from travis
* adding sed for config file
* fix sed
* rm &
* cat redis.conf
* using local redis.conf
* # protected-mode yes
* # supervised no
* redis-server --port 7777 &
* Fix 1293 (#2291)
* fix#1293
* flake8
* fix errors
* addressing @ElDeveloper comments
* rm redis.conf
* fix awaiting_approval list bug
* Fix#2276 (#2294)
* Fix#2276
* Factoring out generate nginx directory file list
* Factoring out the nginx file list writing
* Factoring out generating the file list of an artifact
* Factoring out the header setting
* Addressing @antgonza's comment
* Addressing @wasade's comments
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+22-16Lines changed: 22 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
-
#Contributing to Qiita
1
+
#Contributing to Qiita
2
2
3
3
Qiita is an open source software package, and we welcome community contributions. You can find the source code and test code for Qiita under public revision control in the Qiita git repository on [GitHub](http://github.com/biocore/qiita). We very much welcome contributions.
4
4
5
5
This document covers what you should do to get started with contributing to Qiita. You should read this whole document before considering submitting code to Qiita. This will save time for both you and the Qiita developers.
6
6
7
-
#General Notes on Development
7
+
#General Notes on Development
8
8
9
9
Adding source code to Qiita, can take place in three different modules:
10
10
@@ -17,7 +17,15 @@ always take into consideration how these new features affect users and whether
17
17
or not adding a new section or document to the documentation (found under the
18
18
`doc` folder) would be useful.
19
19
20
-
###The Qiita development rules
20
+
### Repository branch structure
21
+
22
+
The Qiita repository contains three branches:
23
+
24
+
*`master`: This branch reflects the code deployed in our [main Qiita server](http://qiita.microbio.me).
25
+
*`dev`: This branch is the active development branch. All new Pull Requests should be performed against this branch.
26
+
*`release-candidate`: This branch is used to freeze the code from the `dev` branch, so we can deploy in our test servers and exercise the code extensively before deploying in our main system. Code freezes typically occur one week before the scheduled deployment. Check our [milestones page](https://github.com/biocore/qiita/milestones) to see the scheduled deployments.
27
+
28
+
### The Qiita development rules
21
29
22
30
Since Qiita is a package that is continuously growing, we found ourselves in a position where development rules needed to be established so we can reduce both development and reviewer time. These rules are:
23
31
@@ -46,49 +54,47 @@ Since Qiita is a package that is continuously growing, we found ourselves in a p
46
54
9. Last but not least, you are working as part of a team and you should try to help others when possible.
47
55
48
56
49
-
###Configuration file
57
+
###Configuration file
50
58
51
-
The Qiita configuration file determines how the package interacts with your system’s resources (redis, postgres and the IPython cluster). Thus you should review the documentation detailed [here](https://docs.google.com/document/d/1u7kwLP31NM513-8xwpwvLbSQxYu0ehI6Jau1APR13e0/edit#), but especially bear in mind the following points:
59
+
The Qiita configuration file determines how the package interacts with your system’s resources (redisand postgres). Thus you should review the documentation detailed [here](https://docs.google.com/document/d/1u7kwLP31NM513-8xwpwvLbSQxYu0ehI6Jau1APR13e0/edit#), but especially bear in mind the following points:
52
60
53
61
* An example version of this file can be found here `qiita_core/support_files/qiita_config.txt` and if you don’t set a `QIITA_CONFIG_FP` environment variable, that’s the file that Qiita will use.
54
62
* The `[main]` section sets a `TEST_ENVIRONMENT` variable, which determines whether your system will be running unit tests or if it a demo/production system. You will want to set the value to TRUE if you are running the unit tests.
55
63
56
64
**A note on data accumulation**: Qiita keeps data in the `BASE_DATA_DIR` as the system gets used. When you drop a Qiita environment and create a fresh testing environment, the “old” data that was generated from the previous environment should be **manually** deleted (or, at least, removed from the data directories in the `BASE_DATA_DIR`).
57
65
58
-
###Unit tests
66
+
###Unit tests
59
67
60
68
Unit tests in Qiita are located inside the tests/test folder of every sub-module, for example `qiita_db/test/test_metadata_template.py`. These can be executed on a per-file basis or using `nosetests` from the base directory.
61
69
62
70
During test creation make sure the test class is decorated with `@qiita_test_checker()` if database modifications are done during tests. This will automatically drop and rebuild the qiita schema after the entire test class has been executed. This requires to all the tests in a single class be independent of each other, so stochastic failures do not occur due to different test order execution.
63
71
64
72
Coverage testing is in effect, so run tests using `nosetests --with-coverage [test_file.py]` to check what lines of new code in your pull request are not tested.
65
73
66
-
###Documentation
74
+
###Documentation
67
75
68
76
The documentation for Qiita is maintained as part of this repository, under the
69
77
`qiita_pet/support_files/doc` folder, for more information, see the README.md
70
78
file in `qiita_pet/support_files/doc/README.md`.
71
79
72
-
###Scripts
80
+
###Scripts
73
81
74
82
Scripts in Qiita are located inside the scripts directory, their actions will rely on the settings described in the Qiita config file, for example if you are dropping a database, the database that will be dropped is the one described by the `DATABASE` setting. The following is a list of the most commonly used commands during development:
75
83
76
84
*`qiita-env make` will create a new environment (as specified by the Qiita config file).
77
85
*`qiita-env drop` will delete the environment (as specified by the Qiita config file).
78
-
*`qiita-env start_cluster qiita-general`, starts an IPython cluster named ‘qiita-general’. Normally you’ll want to wait a few seconds for the engines to start and become responsive (30-40 seconds depending on your system).
79
-
*`qiita-env stop_cluster qiita-general`, terminates a cluster named ‘qiita-general’.
80
86
*`qiita pet webserver start`, will start the Qiita web-application running on port 21174, you can change this using the `--port` flag, for example `--port=7532`.
81
87
82
-
##Making Database Changes
88
+
##Making Database Changes
83
89
After the initial production release of Qiita, changes to the database schema will require patches; the database can no longer be dropped and recreated using the most recent schema because all the data would be lost! Therefore, patches must be applied instead.
84
90
85
-
###Approach
91
+
###Approach
86
92
87
93
1. We keep "unpatched" versions of the SQL and DBS files in the repository
88
94
2. We keep fully patched versions of the DBS and HTML files in the repository
89
95
3. We keep a patch file for each patch as required in the `qiita_db/support_files/patches` directory. Note that **the patches will be applied in order based on the natural sort order of their filename** (e.g., `2.sql` will be applied before `10.sql`, and `10.sql` will be applied before `a.sql`)
90
96
91
-
###Developer Workflow
97
+
###Developer Workflow
92
98
93
99
1. Load the fully patched DBS file (e.g., `qiita-db.dbs`) in [DBSchema](http://www.dbschema.com/)
94
100
2. Make desired changes
@@ -101,17 +107,17 @@ After the initial production release of Qiita, changes to the database schema wi
101
107
102
108
One drawback is that developers will need to have [DBSchema](http://www.dbschema.com/) to develop for this project.
103
109
104
-
###Data Patches
110
+
###Data Patches
105
111
106
112
If you need to submit a patch that changes only data but does not alter the schema, you should still create a patch file with the next name (e.g., `2.sql`) with your changes. Note that a patch should *not* be created if the modifications do not apply to Qiita databases in general; data patches are only necessary in some cases, e.g., if the terms in an ontology change.
107
113
108
-
###Python Patches
114
+
###Python Patches
109
115
110
116
Occasionally, SQL alone cannot effect the desired changes, and a corresponding python script must be run after the SQL patch is applied. If this is the case, a python file should be created in the `patches/python_patches` directory, and it should have the same basename as the SQL file. For example, if there is a patch `4.sql` in the `patches` directory, and this patch requires a python script be run after the SQL is applied, then the python file should be placed at `patches/python_patches/4.py`. Note that not every SQL patch will have a corresponding python patch, but every python patch will have a corresponding SQL patch.
111
117
112
118
If in the future we discover a use-case where a python patch must be applied for which there *is no corresponding SQL patch*, then a blank SQL patch file will still need to be created.
113
119
114
-
##SQL coding guidelines
120
+
##SQL coding guidelines
115
121
Since the `qiita_db` code contains a mixture of python code and SQL code, here are some coding guidelines to add SQL code to Qiita:
Copy file name to clipboardExpand all lines: INSTALL.md
+27-16Lines changed: 27 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -60,6 +60,7 @@ Install the non-python dependencies
60
60
61
61
*[PostgreSQL](http://www.postgresql.org/download/) (minimum required version 9.3.5, we have tested most extensively with 9.3.6)
62
62
*[redis-server](http://redis.io) (we have tested most extensively with 2.8.17)
63
+
*[webdis] (https://github.com/nicolasff/webdis) (latest version should be fine but we have tested the most with 9ee6fe2 - Feb 6, 2016)
63
64
64
65
There are several options to install these dependencies depending on your needs:
65
66
@@ -87,6 +88,27 @@ brew update
87
88
brew install homebrew/versions/redis28
88
89
```
89
90
91
+
### webdis
92
+
93
+
Note that this is the only package that assumes that Qiita is already installed (due to library dependencies). Also, that the general suggestion is to have 2 redis servers running, one for webdis/redbiom and the other for Qiita. The reason for multiple redis servers is so that the redbiom cache can be flushed without impacting the operation of the qiita server itself.
94
+
95
+
The following instructions install, compile and pre-populates the redbiom redis DB so we assume that redis is running on the default port and that Qiita is fully installed as the redbiom package is installed with Qiita.
Finally, redbiom relies on the REDBIOM_HOST environment variable to set the URL to query. By default is set to Qiita redbiom public repository. To change it you could do:
177
+
161
178
162
179
```bash
163
-
qiita-env make --no-load-ontologies
180
+
export REDBIOM_HOST=http://my_host.com:7379
164
181
```
165
182
166
183
## Start Qiita
@@ -173,12 +190,6 @@ Next, start redis server (the command may differ depending on your operating sys
0 commit comments