Mark 2.0.0 (#757)

* Update CHANGELOG
* Update README
* Upgrade migration script to port visibility settings
* Add message about visibility settings and port over visibility settings
* Close #758
* Add tests for dynamic value challenges

CHANGELOG.md

@@ -1,21 +1,40 @@
-2.0.0 / 2018-11-22
+2.0.0 / 2018-11-26
 ==================
 
-2.0.0 is a significant change. 
+2.0.0 is a *significant*, backwards-incompaitble release. 
-If you are upgrading from a prior version be sure to make backups before upgrading.
 
-In addition, 2.0.0 is a backwards-incompatible release. Many plugins will be broken in CTFd 2.0.0, and if you're having 
+Many unofficial plugins will not be supported in CTFd 2.0.0. If you're having trouble updating your plugins 
-trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io/) for help and discussion.
+please join [the CTFd Slack](https://slack.ctfd.io/) for help and discussion.
+
+If you are upgrading from a prior version be sure to make backups and have a reversion plan before upgrading.  
+
+* If upgrading from 1.2.0 please make use of the `migrations/1_2_0_upgrade_2_0_0.py` script as follows:
+    1. Make all necessary backups. Backup the database, uploads folder, and source code directory.
+    2. Upgrade the source code directory (i.e. `git pull`) but do not run any updated code yet. 
+    3. Set the `DATABASE_URL` in `CTFd/config.py` to point to your existing CTFd database.
+    3. Run the upgrade script from the CTFd root folder i.e. `python migrations/1_2_0_upgrade_2_0_0.py`.
+        * This migration script will attempt to migrate data inside the database to 2.0.0 but it cannot account for every situation. 
+        * Examples of situations where you may need to manually migrate data:
+            * Tables/columns created by plugins
+            * Tables/columns created by forks
+            * Using databases which are not officially supported (e.g. sqlite, postgres)
+    4. Setup the rest of CTFd (i.e. config.py), migrate/update any plugins, and run normally.
+* If upgrading from a version before 1.2.0, please upgrade to 1.2.0 and then continue with the steps above. 
 
 **General**
 
 * Seperation of Teams into Users and Teams.
-* Integration with MajorLeagueCyber. (https://majorleaguecyber.org)
+    * Use User Mode if you want users to register as themselves and play on their own.
+    * Use Team Mode if you want users to create and join teams to play together.
+* Integration with MajorLeagueCyber (MLC). (https://majorleaguecyber.org)
+    * Organizers can register their event with MLC and will receive OAuth Client ID & Client Secret.
+    * Organizers can set those OAuth credentials in CTFd to allow users and teams to automatically register in a CTF. 
 * Data is now provided to the front-end via the REST API. (#551)
     * Javascript uses `fetch()` to consume the REST API.
-* Dynamic Challenges built in. 
+* Dynamic Challenges are built in. 
-* S3 Uploader built in. (#661)
+* S3 backed uploading/downloading built in. (#661)
-* Real time notifications. (#600)
+* Real time notifications/announcements. (#600)
+    * Uses long-polling instead of websockets to simplify deployment.
 * Email address domain whitelisting. (#603)
 * Database exporting to CSV. (#656)
 * Imports/Exports rewritten to act as backups.
@@ -27,6 +46,7 @@ trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io
     * Based on https://github.com/umpirsky/country-list/blob/master/data/en_US/country.csv.
 * Sessions are no longer stored using secure cookies. (#658)
     * Sessions are now stored server side in a cache (`filesystem` or `redis`) allowing for session revocation.
+    * In order to delete the cache during local development you can delete `CTfd/.data/filesystem_cache`.
 * Challenges can now have requirements which must be met before the challenge can be seen/solved.
 * Workshop mode, score hiding, registration hiding, challenge hiding have been changed to visibility settings.
 * Users and Teams can now be banned preventing access to the CTF.
@@ -34,13 +54,8 @@ trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io
     * WORKERS count in `docker-entrypoint.sh` defaults to 1. (#716)
     * `docker-entrypoint.sh` exits on any error. (#717)
 * Increased test coverage.
+* Create `SAFE_MODE` configuration to disable loading of plugins.
 * Migrations have been reset. 
-    * If upgrading from 1.2.0:
-        1. Make all necessary backups. Backup the database, uploads folder, and source code directory.
-        2. Upgrade the source code directory.
-        3. Set the `DATABASE_URL` in `CTFd/config.py`.
-        3. Run the upgrade script from the CTFd folder i.e. `python migrations/1_2_0_upgrade_2_0_0.py`.
-        4. Setup the rest of CTFd and run normally.
 
 **Themes**
 
@@ -48,24 +63,27 @@ trouble updating your plugins please join [the CTFd Slack](https://slack.ctfd.io
     * Javascript uses `fetch()` to consume the REST API.
 * The admin theme is no longer considered seperated from the core theme and should always be together.
 * Themes now use `url_for()` to generate URLs instead of hardcoding.
-* socket.io is used to connect to CTFd to receive notifications.
+* socket.io (via long-polling) is used to connect to CTFd to receive notifications.
 * `ctf_name()` renamed to `get_ctf_name()` in themes.
 * `ctf_logo()` renamed to `get_ctf_logo()` in themes.
 * `ctf_theme()` renamed to `get_ctf_theme()` in themes.
 * Update Font-Awesome to 5.4.1.
 * Update moment.js to 2.22.2. (#704)
+* Workshop mode, score hiding, registration hiding, challenge hiding have been changed to visibility functions.
+    * `accounts_visible()`, `challenges_visible()`, `registration_visible()`, `scores_visible()`
 
 **Plugins**
 
 * Plugins are loaded in `sorted()` order
-* Rename challenge type plugins to use .html and have simplified names. (create, update, view)
+* Rename challenge type plugins to use `.html` and have simplified names. (create, update, view)
-* Many functions moved around because utils.py has been broken up and refactored. (#475)
+* Many functions have moved around because utils.py has been broken up and refactored. (#475)
-* Marshmallow (https://marshmallow.readthedocs.io) is now used by the REST API to validate and serialize/deserialize data.
+* Marshmallow (https://marshmallow.readthedocs.io) is now used by the REST API to validate and serialize/deserialize API data.
-    * Marshmallow schemas and views are used to restrict SQLAlchemy columns to user types. 
+    * Marshmallow schemas and views are used to restrict SQLAlchemy columns to user roles. 
 * The REST API features swagger support but this requires more utilization internally.
 * Errors can now be provided between routes and decoraters through message flashing. (CTFd.utils.helpers; get_errors, get_infos, info_for, error_for)
 * Email registration regex relaxed. (#693)
 * Many functions have moved and now have dedicated utils packages for their category.
+* Create `SAFE_MODE` configuration to disable loading of plugins.
 
 
 1.2.0 / 2018-05-04

CTFd/plugins/dynamic_challenges/__init__.py

@@ -2,7 +2,7 @@ from __future__ import division  # Use floating point for math calculations
 from CTFd.plugins.challenges import BaseChallenge, CHALLENGE_CLASSES
 from CTFd.plugins import register_plugin_assets_directory
 from CTFd.plugins.flags import get_flag_class
-from CTFd.models import db, Solves, Fails, Flags, Challenges, Files, Tags, Teams, Hints
+from CTFd.models import db, Solves, Fails, Flags, Challenges, ChallengeFiles, Tags, Teams, Hints
 from CTFd import utils
 from CTFd.utils.migrations import upgrade
 from CTFd.utils.user import get_ip
@@ -122,10 +122,10 @@ class DynamicValueChallenge(BaseChallenge):
         Fails.query.filter_by(challenge_id=challenge.id).delete()
         Solves.query.filter_by(challenge_id=challenge.id).delete()
         Flags.query.filter_by(challenge_id=challenge.id).delete()
-        files = Files.query.filter_by(challenge_id=challenge.id).all()
+        files = ChallengeFiles.query.filter_by(challenge_id=challenge.id).all()
         for f in files:
             delete_file(f.id)
-        Files.query.filter_by(challenge_id=challenge.id).delete()
+        ChallengeFiles.query.filter_by(challenge_id=challenge.id).delete()
         Tags.query.filter_by(challenge_id=challenge.id).delete()
         Hints.query.filter_by(challenge_id=challenge.id).delete()
         DynamicChallenge.query.filter_by(id=challenge.id).delete()

CTFd/themes/admin/templates/configs/settings.html

@@ -38,6 +38,9 @@
 					Admins Only
 				</option>
 			</select>
+			<small class="form-text text-muted">
+				This setting should generally be the same as Account Visibility to avoid conflicts.
+			</small>
 		</div>
 
 		<div class="form-group">
@@ -58,6 +61,9 @@
 					Admins Only
 				</option>
 			</select>
+			<small class="form-text text-muted">
+				This setting should generally be the same as Score Visibility to avoid conflicts.
+			</small>
 		</div>
 
 		<div class="form-group">

CTFd/utils/updates/__init__.py

@@ -36,7 +36,7 @@ def update_check(force=False):
 
     if update:
         try:
-            name = get_config('ctf_name') or ''
+            name = str(get_config('ctf_name')) or ''
             params = {
                 'ctf_id': sha256(name),
                 'current': app.VERSION,

README.md

@@ -15,7 +15,8 @@ CTFd is a Capture The Flag framework focusing on ease of use and customizability
     * Unlockable challenge support
     * Challenge plugin architecture to create your own custom challenges
     * Static & Regex based flags
-    * Users can unlock hints for free or with points
+        * Custom flag plugins
+    * Unlockable hints
     * File uploads to the server or an Amazon S3-compatible backend
     * Limit challenge attempts & hide challenges
     * Automatic bruteforce protection
@@ -57,7 +58,7 @@ https://demo.ctfd.io/
 ## Support
 To get basic support, you can join the [CTFd Slack Community](https://slack.ctfd.io/): [![CTFd Slack](https://slack.ctfd.io/badge.svg)](https://slack.ctfd.io/)
 
-If you prefer commercial support or have a special project, send us an email: [support@ctfd.io](mailto:support@ctfd.io).
+If you prefer commercial support or have a special project, feel free to [contact us](https://ctfd.io/contact/).
 
 ## Managed Hosting
 Looking to use CTFd but don't want to deal with managing infrastructure? Check out [the CTFd website](https://ctfd.io/) for managed CTFd deployments.
@@ -67,7 +68,7 @@ CTFd is heavily integrated with [MajorLeagueCyber](https://majorleaguecyber.org/
 
 By registering your CTF event with MajorLeagueCyber users can automatically login, track their individual and team scores, submit writeups, and get notifications of important events. 
 
-To integrate with MajorLeagueCyber, simply register an account, create an event, and install the client ID and client secret in the relevant portion in CTFd/config.py:
+To integrate with MajorLeagueCyber, simply register an account, create an event, and install the client ID and client secret in the relevant portion in `CTFd/config.py` or in the admin panel:
 
 ```python
 OAUTH_CLIENT_ID = None

migrations/1_2_0_upgrade_2_0_0.py

@@ -168,16 +168,70 @@ if __name__ == '__main__':
     print('MIGRATING Config')
     banned = [
         'ctf_version',
-        'setup'
     ]
+    workshop_mode = None
+    hide_scores = None
+    prevent_registration = None
+    view_challenges_unregistered = None
+    view_scoreboard_if_authed = None
+
+    challenge_visibility = 'private'
+    registration_visibility = 'public'
+    score_visibility = 'public'
+    account_visibility = 'public'
     for config in old_data['config']:
         config.pop('id')
+
+        if config['key'] == 'workshop_mode':
+            workshop_mode = config['value']
+        elif config['key'] == 'hide_scores':
+            hide_scores = config['value']
+        elif config['key'] == 'prevent_registration':
+            prevent_registration = config['value']
+        elif config['key'] == 'view_challenges_unregistered':
+            view_challenges_unregistered = config['value']
+        elif config['key'] == 'view_scoreboard_if_authed':
+            view_scoreboard_if_authed = config['value']
+
         if config['key'] not in banned:
             new_conn['config'].insert(dict(config))
+
+    if workshop_mode:
+        score_visibility = 'admins'
+        account_visibility = 'admins'
+
+    if hide_scores:
+        score_visibility = 'hidden'
+
+    if prevent_registration:
+        registration_visibility = 'private'
+
+    if view_challenges_unregistered:
+        challenge_visibility = 'public'
+
+    if view_scoreboard_if_authed:
+        score_visibility = 'private'
+
     new_conn['config'].insert({
         'key': 'user_mode',
         'value': 'users'
     })
+    new_conn['config'].insert({
+        'key': 'challenge_visibility',
+        'value': challenge_visibility
+    })
+    new_conn['config'].insert({
+        'key': 'registration_visibility',
+        'value': registration_visibility
+    })
+    new_conn['config'].insert({
+        'key': 'score_visibility',
+        'value': score_visibility
+    })
+    new_conn['config'].insert({
+        'key': 'account_visibility',
+        'value': account_visibility
+    })
     del old_data['config']
 
     manual = []
@@ -208,6 +262,7 @@ if __name__ == '__main__':
     for table in manual:
         print('\t', 'ALTER TABLE `{table}` ADD PRIMARY KEY(id)'.format(table=table))
 
-    print('The following tables were not created because they were empty and must be manually recreated (e.g. app.db.create_all()')
+    print(
+        'The following tables were not created because they were empty and must be manually recreated (e.g. app.db.create_all()')
     for table in not_created:
         print('\t', table)

tests/challenges/test_dynamic.py

@@ -0,0 +1,116 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+from CTFd.plugins.dynamic_challenges import DynamicChallenge, DynamicValueChallenge
+from tests.helpers import *
+
+
+def test_can_create_dynamic_challenge():
+    """Test that dynamic challenges can be made from the API/admin panel"""
+    app = create_ctfd(enable_plugins=True)
+    with app.app_context():
+        register_user(app)
+        client = login_as_user(app, name="admin", password="password")
+
+        challenge_data = {
+            "name": "name",
+            "category": "category",
+            "description": "description",
+            "value": 100,
+            "decay": 20,
+            "minimum": 1,
+            "state": "hidden",
+            "type": "dynamic"
+        }
+
+        r = client.post('/api/v1/challenges', json=challenge_data)
+        assert r.get_json().get('data')['id'] == 1
+
+        challenges = DynamicChallenge.query.all()
+        assert len(challenges) == 1
+
+        challenge = challenges[0]
+        assert challenge.value == 100
+        assert challenge.initial == 100
+        assert challenge.decay == 20
+        assert challenge.minimum == 1
+    destroy_ctfd(app)
+
+
+def test_can_update_dynamic_challenge():
+    """Test that dynamic challenges can be deleted"""
+    app = create_ctfd(enable_plugins=True)
+    with app.app_context():
+        challenge_data = {
+            "name": "name",
+            "category": "category",
+            "description": "description",
+            "value": 100,
+            "decay": 20,
+            "minimum": 1,
+            "state": "hidden",
+            "type": "dynamic"
+        }
+        req = FakeRequest(form=challenge_data)
+        challenge = DynamicValueChallenge.create(req)
+
+        assert challenge.value == 100
+        assert challenge.initial == 100
+        assert challenge.decay == 20
+        assert challenge.minimum == 1
+
+        challenge_data = {
+            "name": "new_name",
+            "category": "category",
+            "description": "new_description",
+            "value": "200",
+            "initial": "200",
+            "decay": "40",
+            "minimum": "5",
+            "max_attempts": "0",
+            "state": "visible"
+        }
+
+        req = FakeRequest(form=challenge_data)
+        challenge = DynamicValueChallenge.update(challenge, req)
+
+        assert challenge.name == 'new_name'
+        assert challenge.description == "new_description"
+        assert challenge.value == 200
+        assert challenge.initial == 200
+        assert challenge.decay == 40
+        assert challenge.minimum == 5
+        assert challenge.state == "visible"
+
+    destroy_ctfd(app)
+
+
+def test_can_delete_dynamic_challenge():
+    app = create_ctfd(enable_plugins=True)
+    with app.app_context():
+        register_user(app)
+        client = login_as_user(app, name="admin", password="password")
+
+        challenge_data = {
+            "name": "name",
+            "category": "category",
+            "description": "description",
+            "value": 100,
+            "decay": 20,
+            "minimum": 1,
+            "state": "hidden",
+            "type": "dynamic"
+        }
+
+        r = client.post('/api/v1/challenges', json=challenge_data)
+        assert r.get_json().get('data')['id'] == 1
+
+        challenges = DynamicChallenge.query.all()
+        assert len(challenges) == 1
+
+        challenge = challenges[0]
+        DynamicValueChallenge.delete(challenge)
+
+        challenges = DynamicChallenge.query.all()
+        assert len(challenges) == 0
+    destroy_ctfd(app)

tests/helpers.py

@@ -4,6 +4,7 @@ from CTFd.models import *
 from CTFd.cache import cache
 from sqlalchemy_utils import database_exists, create_database, drop_database
 from sqlalchemy.engine.url import make_url
+from collections import namedtuple
 import datetime
 import six
 import gc
@@ -16,6 +17,9 @@ else:
     binary_type = bytes
 
 
+FakeRequest = namedtuple('FakeRequest', ['form'])
+
+
 def create_ctfd(ctf_name="CTFd", name="admin", email="admin@ctfd.io", password="password", user_mode="users", setup=True, enable_plugins=False):
     if enable_plugins:
         TestingConfig.SAFE_MODE = False