diff options
| author |  vnugent <public@vaughnnugent.com> | 2024-06-24 17:34:40 -0400 |
|---|---|---|
| committer | vnugent <public@vaughnnugent.com> | 2024-06-24 17:34:40 -0400 |
| commit | 18e6823064f78d5821801839882d5fa3dadf79e3 (patch) | |
| tree | 0bbd1baf2492fa3df5197dd82640592ba7b73c50 /ci | |
| parent | 71b581e006fda94aa675b7df9a7fbfe35d748b48 (diff) | |
Squashed commit of the following:
commit 5dc6decde6f86ba87173a1e10b0de8153999090f
Author: vnugent <public@vaughnnugent.com>
Date: Mon Jun 24 17:05:40 2024 -0400
fix ci main plugin name
commit a355e12201f50d8f52738d225c270042913030e2
Author: vnugent <public@vaughnnugent.com>
Date: Mon Jun 24 16:46:04 2024 -0400
ci: Consolidate ci configuration
commit 86d6ba0c9869f21b2b16728abf4cb20fcf6ed769
Merge: bbe3b9b 71b581e
Author: vnugent <public@vaughnnugent.com>
Date: Tue Jun 4 16:00:01 2024 -0400
Merge branch 'master' into develop
commit bbe3b9b80db68cf86e26bd2e40a07c1650031224
Author: vnugent <public@vaughnnugent.com>
Date: Tue Jun 4 15:54:42 2024 -0400
ignore npm version bump errors for build
commit 45816924e5a47710a6bc4ed0d59ea81a48eddd1e
Author: vnugent <public@vaughnnugent.com>
Date: Tue Jun 4 15:30:35 2024 -0400
project version in ui
commit 03357c9f7b7fa389d2d426e95d9854b7b04623f9
Author: vnugent <public@vaughnnugent.com>
Date: Tue Jun 4 12:59:30 2024 -0400
chore: Essentials upstream update
commit b6d292014700d05a93aa4e486baedf37656f464a
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 23 19:09:28 2024 -0400
docs: Update readme instructions
commit 5b6b4c06bc4b974e1839ca47c91dd6c903f119fd
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 23 18:17:56 2024 -0400
build(server): Update Docker config to support new vars
commit c7c9e8a441e99e1dc79e2a690d83281af463d817
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 23 17:40:09 2024 -0400
update backend deps
commit 06eb12d107f2605cae0f14884de04058c02d29f7
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 23 17:38:22 2024 -0400
chore: package updates and verify
commit c8e3ca86be45be05c1f76a7dd808275a1afaccb0
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 16 17:10:27 2024 -0400
chore: Remove unused cache stuff
commit ba2ee1073a5042bda0afc2b56233fe99d491b39d
Merge: e6b4e60 28e67e5
Author: vnugent <public@vaughnnugent.com>
Date: Wed May 15 18:08:48 2024 -0400
Merge branch 'master' into develop
commit e6b4e605622f2b323fd3fe0aa01000f3986e3bba
Author: vnugent <public@vaughnnugent.com>
Date: Wed May 15 16:35:06 2024 -0400
build: Fix ssl container dir, and certificate automation
commit a710553aa5c0484b6934b8a5d4d16b1eb3ff2a88
Author: vnugent <public@vaughnnugent.com>
Date: Wed May 15 15:37:53 2024 -0400
ci: Admin form clear, polish build and release taskfiels
commit 372eb6a32a9891afb5035caa0805e8ed878416d8
Author: vnugent <public@vaughnnugent.com>
Date: Tue May 14 22:27:10 2024 -0400
default read secret pepper from file for bare-metal builds
commit 1b7270b40b65ef089bf40a14065227f742b8507e
Author: vnugent <public@vaughnnugent.com>
Date: Tue May 14 21:52:03 2024 -0400
chore: make container slightly easier to setup
commit 2deda50a167286bc93fd3871a1fd6dbf9f43c81f
Merge: 28f0f77 96ae7b0
Author: vnugent <public@vaughnnugent.com>
Date: Sun Apr 28 10:54:09 2024 -0400
Merge branch 'master' into develop
commit 28f0f774da975c04271445761b2de31aecf969ff
Author: vnugent <public@vaughnnugent.com>
Date: Sat Apr 27 23:11:48 2024 -0400
Add --no-cache arg to build image args
commit 22a1f5d374ec1a487944c6303066d0f15617cb12
Author: vnugent <public@vaughnnugent.com>
Date: Sat Apr 27 18:47:25 2024 -0400
fix: Upstream patch missing ! in middlware
commit f40ca2d4c26f81276d58760152592a918bf3cd87
Author: vnugent <public@vaughnnugent.com>
Date: Sat Apr 27 18:36:01 2024 -0400
chore: package updates
commit d7a3c957467e65ea7176170fba3c280ac18ac17e
Author: vnugent <public@vaughnnugent.com>
Date: Sun Apr 21 12:02:57 2024 -0400
chore: Package updates and minor QOL patches
commit 97a5bded5122708cf39d0e86bc24a5f31755bdd1
Merge: 56e0a38 5877c86
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 9 17:38:02 2024 -0400
Merge branch 'master' into develop
commit 56e0a38b2ca246e8beeaef3c6c4b9c0ce7d0f09b
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 9 17:35:13 2024 -0400
chore(app): Update deps, login spinner, curl msg, view prep
commit 0945210c0492dd8a8de99ccd8e5e66cf05e3a1c1
Merge: 24fac82 3c15d54
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 2 14:58:59 2024 -0400
Merge branch 'master' into develop
commit 24fac82efe9e5c18e86ed535678640e7401472db
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 2 14:54:20 2024 -0400
ci: Configure manual dep versions
commit d2ae31ec919d72e66d8b40db8394b55efd6ea6d3
Author: vnugent <public@vaughnnugent.com>
Date: Sun Mar 31 22:19:53 2024 -0400
ci: Native compression support for win
commit fa7fdef79c6d468022b77f81314ac129fe0cdc32
Merge: 308092d a01220a
Author: vnugent <public@vaughnnugent.com>
Date: Wed Mar 13 21:26:55 2024 -0400
Merge branch 'master' into develop
commit 308092d6d743d0ba8f7ca86fd77e9c837dc46e88
Merge: 48637a8 9134093
Author: vnugent <public@vaughnnugent.com>
Date: Wed Mar 13 21:01:02 2024 -0400
Merge branch 'master' into develop
commit 48637a8781fc951c307216f604fc1610e68691c3
Merge: 1e08c6d e326736
Author: vnugent <public@vaughnnugent.com>
Date: Wed Mar 13 16:20:35 2024 -0400
Merge branch 'master' into develop
commit 1e08c6d2112459dc02a0ab873123c4a363b01d21
Author: vnugent <public@vaughnnugent.com>
Date: Wed Mar 13 16:17:58 2024 -0400
ci: verified container build ready for next release
commit 85a1e5b7cc5c99e97a2d4e99bbceb0d2139742ff
Author: vnugent <public@vaughnnugent.com>
Date: Tue Mar 12 22:05:16 2024 -0400
ci: exciting bare-metal build process, os support, smaller packages
commit 748cdbf4880d830fd794e92856e8c35a46e4f884
Author: vnugent <public@vaughnnugent.com>
Date: Mon Mar 11 21:21:18 2024 -0400
feat(app): #1 update libs & add curl support
Diffstat (limited to 'ci')
23 files changed, 161 insertions, 422 deletions
diff --git a/ci/build.env b/ci/build.env new file mode 100644 index 0000000..b0933f0 --- /dev/null +++ b/ci/build.env @@ -0,0 +1,76 @@ +# +# Copyright (C) Vaughn Nugent +# +# This file contains static variables that are substituded at build time +# to the configuration templates for bare-metal packages as a set of default +# variables. This is so I can use the same template files for container builds +# and bare-metal +# + +########## +# _internal_ +########## + +#default compression lib for bare metal +COMPRESSION_LIB_PATH=lib/vnlib_compress.dll + +########## +# Plugins +########## + +DEBUG_PLUGINS=false +MAX_LOGIN_ATTEMPS=10 +MAX_BOOKMARKS=5000 +REG_TOKEN_DURATION_MIN=360 + +########## +# HTTP +########## + +#sets the absolute maxium upload size +MAX_CONTENT_LENGTH=5120000 + +HTTP_TRACE_ON=false +HTTP_DOWNSTREAM_SERVERS=[] + +#default ssl dir is ssl/ +SSL_JSON={"cert": "ssl/cert.pem", "privkey":"ssl/key.pem"} + +############# +# HC VAULT +############# + +HC_VAULT_ADDR= +HC_VAULT_TOKEN= +HC_VAULT_TRUST_CERT=false + +########## +# SQL +########## + +#default to sqlite as that is whats packaged by default, also use the data/ directory +SQL_LIB_PATH=VNLib.Plugins.Extensions.Sql.SQLite.dll +SQL_CONNECTION_STRING=Data Source=data/simple-bookmark.db; + +############## +# CACHE +############## + +CACHE_ASM_PATH=VNLib.Data.Caching.Providers.VNCache.dll +REDIS_CONNECTION_STRING= +VNCACHE_INITIAL_NODES=[] + +#for self contained apps, memcache needs to be true +MEMCACHE_ONLY=true + +############ +# SECRETS +############ + +PASSWORD_PEPPER=file://secrets/password-pepper.txt +DATABASE_PASSWORD= +REDIS_PASSWORD= + +#no cache, so clear vars +VNCACHE_CLIENT_PRIVATE_KEY= +VNCACHE_CACHE_PUBLIC_KEY=
\ No newline at end of file diff --git a/ci/compile.ps1 b/ci/compile.ps1 new file mode 100644 index 0000000..2858d06 --- /dev/null +++ b/ci/compile.ps1 @@ -0,0 +1,23 @@ +param([String] $inputDir, [String] $outputDir) + +$templateFiles = Get-ChildItem -Path $inputDir -Filter "*-template.json" -File + +foreach ($file in $templateFiles) { + $baseFilename = $file.BaseName + '.json' + $templateFilePath = $file.FullName + + #remove the -template.json suffix + $outputFilePath = Join-Path -Path $outputDir -ChildPath $baseFilename.replace("-template","") + + #substitute environment variables for file variables + Get-Content $templateFilePath | ForEach-Object { + if ($_ -match "\$\{((\w+))\}") + { + $_ -replace "\$\{(\w+)\}",$([Environment]::GetEnvironmentVariable($Matches[1])) + } + else + { + $_ + } + } | Set-Content $outputFilePath +}
\ No newline at end of file diff --git a/ci/container/config-templates/Essentials.Accounts-template.json b/ci/config-templates/Essentials.Accounts-template.json index 68568a9..54e9b58 100644 --- a/ci/container/config-templates/Essentials.Accounts-template.json +++ b/ci/config-templates/Essentials.Accounts-template.json @@ -5,8 +5,8 @@ "login_endpoint": { "path": "/api/account/login", - "max_login_attempts": ${MAX_LOGIN_ATTEMPS}, //10 failed attempts in 10 minutes - "failed_attempt_timeout_sec": 600 //10 minutes + "max_login_attempts": ${MAX_LOGIN_ATTEMPS}, //10 failed attempts in 10 minutes + "failed_attempt_timeout_sec": 600 //10 minutes }, "keepalive_endpoint": { diff --git a/ci/config-templates/PageRouter-template.json b/ci/config-templates/PageRouter-template.json new file mode 100644 index 0000000..86a51f8 --- /dev/null +++ b/ci/config-templates/PageRouter-template.json @@ -0,0 +1,7 @@ +{ + "debug": ${DEBUG_PLUGINS}, + "store": { + //All builds require the routes.xml file in the config directory even after variable substitution + "route_file": "config/routes.xml" + } +}
\ No newline at end of file diff --git a/ci/container/config-templates/SessionProvider-template.json b/ci/config-templates/SessionProvider-template.json index 328f06f..328f06f 100644 --- a/ci/container/config-templates/SessionProvider-template.json +++ b/ci/config-templates/SessionProvider-template.json diff --git a/ci/container/config-templates/SimpleBookmark-template.json b/ci/config-templates/SimpleBookmark-template.json index 8736d8d..8736d8d 100644 --- a/ci/container/config-templates/SimpleBookmark-template.json +++ b/ci/config-templates/SimpleBookmark-template.json diff --git a/ci/container/config-templates/config-template.json b/ci/config-templates/config-template.json index 0bd0ddb..7055678 100644 --- a/ci/container/config-templates/config-template.json +++ b/ci/config-templates/config-template.json @@ -15,7 +15,7 @@ //The maxium ammount of data (in bytes) allows for mulitpart/form data file uploads "multipart_max_size": 80240, //Absolute maximum size (in bytes) of the request entity body (exludes headers) - "max_entity_size": 1024000, + "max_entity_size": ${MAX_CONTENT_LENGTH}, //Keepalive ms for HTTP1.1 keepalive connections "keepalive_ms": 1000000, //The buffer size to use when parsing headers (also the maxium request header size allowed) @@ -25,7 +25,7 @@ //The maxium number of allowed network connections, before 503s will be issued automatically and connections closed "max_connections": 5000, //The size in bytes of the buffer to use when writing response messages - "response_buf_size": 4096, + "response_buf_size": 65536, //time (in ms) to wait for a response from an active connection in recv mode, before dropping it "recv_timeout_ms": 5000, //Time in ms to wait for the client to accept transport data before terminating the connection @@ -41,7 +41,7 @@ //Setup the native lib "vnlib.net.compression": { - "lib_path": "lib/libvn_compress.so", + "lib_path": "${COMPRESSION_LIB_PATH}", "level": 1 }, @@ -71,7 +71,7 @@ //A list of file extensions to deny access to, if a resource is requested and has one of the following extensions, a 404 is returned "deny_extensions": [ ".ts", ".json", ".htaccess", ".php" ], //The default file extensions to append to a resource that does not have a file extension - "default_files": [ "index.html", "index.htm" ], + "default_files": [ "index.html" ], //A list of error file objects, files are loaded into memory (and watched for changes) and returned when the specified error code occurs "error_files": [], diff --git a/ci/config/routes.xml b/ci/config-templates/routes.xml index 85f9830..85f9830 100644 --- a/ci/config/routes.xml +++ b/ci/config-templates/routes.xml diff --git a/ci/config/Essentials.Accounts.json b/ci/config/Essentials.Accounts.json deleted file mode 100644 index 680f92a..0000000 --- a/ci/config/Essentials.Accounts.json +++ /dev/null @@ -1,76 +0,0 @@ -{ - "debug": false, - - //endpoints - - "login_endpoint": { - "path": "/api/account/login", - "max_login_attempts": 10, //10 failed attempts in 10 minutes - "failed_attempt_timeout_sec": 600 //10 minutes - }, - - "keepalive_endpoint": { - "path": "/api/account/keepalive", - //Regen token every 10 mins along with cookies - "token_refresh_sec": 600 //10 minutes - }, - - "profile_endpoint": { - "path": "/api/account/profile" - }, - - "password_endpoint": { - "path": "/api/account/reset" - }, - - "mfa_endpoint": { - "path": "/api/account/mfa" - }, - - "logout_endpoint": { - "path": "/api/account/logout" - }, - - "pki_auth_endpoint": { - "path": "/api/account/pki", - "jwt_time_dif_sec": 30, - "max_login_attempts": 10, - "failed_attempt_timeout_sec": 600, - //Configures the PATCH and DELETE methods to update the user's stored key when logged in - "enable_key_update": true - }, - - //If mfa is defined, configures mfa enpoints and enables mfa logins - "mfa": { - "upgrade_expires_secs": 180, - "nonce_size": 64, - - //Defines totp specific arguments - "totp": { - "digits": 6, - "issuer": "Simple-Bookmark", - "period_secs": 30, - "algorithm": "sha1", - "secret_size": 32, - "window_size": 2 - } - }, - - //Defines the included account provider - "account_security": { - //Time in seconds before a session is considered expired - "session_valid_for_sec": 3600, - //Path/domain for all security cookies - "cookie_domain": "", - "cookie_path": "/", - "status_cookie_name": "li", //front-end cookie name must match to detect login status - "otp_header_name": "X-Web-Token", //Front-end header name must match - "otp_time_diff_sec": 30, - "otp_key_size": 64, - "pubkey_cookie_name": "client-id", - "pubkey_signing_key_size": 32, - "strict_origin": false, - "strict_path": true, //Can be enabled if front-end is running on the same server - //"allowed_origins": [""] - } -}
\ No newline at end of file diff --git a/ci/config/PageRouter.json b/ci/config/PageRouter.json deleted file mode 100644 index 7d24da8..0000000 --- a/ci/config/PageRouter.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "debug": false, - "store": { - "route_file": "config/routes.xml" - } -}
\ No newline at end of file diff --git a/ci/config/SessionProvider.json b/ci/config/SessionProvider.json deleted file mode 100644 index 6828114..0000000 --- a/ci/config/SessionProvider.json +++ /dev/null @@ -1,25 +0,0 @@ -{ - - "debug": false, - - //Provider assemblies to load - "provider_assemblies": [ "VNLib.Plugins.Sessions.VNCache.dll" ], - - //Web session provider, valid format for VNCache and also memory sessions - "web": { - //Cache system key prefix - "cache_prefix": "websessions", - //The session cookie name - "cookie_name": "sb-session", - //Size in bytes for generated session ids - "cookie_size": 40, - //time (in seconds) a session is valid for - "valid_for_sec": 3600, - //The maxium number of connections waiting for the cache server responses - "max_waiting_connections": 100, - //Enforce strict cross-origin session checks - "strict_cors": true, - ///Enforces strict TLS to help prevent tls downgrades based on stored session variables (privacy note: this can be leaked through brute-forced if session id is stolen) - "strict_tls_protocol": true - } -}
\ No newline at end of file diff --git a/ci/config/SimpleBookmark.json b/ci/config/SimpleBookmark.json deleted file mode 100644 index 2cb64bf..0000000 --- a/ci/config/SimpleBookmark.json +++ /dev/null @@ -1,35 +0,0 @@ -{ - - //Comments are allowed - "debug": false, //Enables obnoxious debug logging - - "bm_endpoint": { - - "path": "/api/bookmarks", //Path for the bookmarks endpoint - - "config": { - "max_limit": 100, //Max results per page - "default_limit": 20, //Default results per page - "user_quota": 5000 //Max bookmarks per user - } - }, - - //System website lookup endpoint (aka curl) - "curl": { - "path": "/api/lookup", - "exe_path": "curl", //Path to the curl executable - "extra_args": [ - "--globoff", //Disables unsafe url globbing - "--no-keepalive", //Disables keepalive, uneeded for a single lookup request - "--max-filesize", "100K", //Max file size 100K - "--max-redirs", "5", //Max redirects 5 - "--location" //Follow redirects - ] - }, - - "registration": { - "path": "/api/register", //Path for the registration endpoint - "token_lifetime_mins": 360, //Token lifetime in minutes - "key_regen_interval_mins": 3600 //Signing key regeneration interval in minutes - } -}
\ No newline at end of file diff --git a/ci/config/config.json b/ci/config/config.json deleted file mode 100644 index f776732..0000000 --- a/ci/config/config.json +++ /dev/null @@ -1,182 +0,0 @@ -{ - - //Host application config, config is loaded as a read-only DOM that is available - //to the host and loaded child plugins, all elements are available to plugins via the 'HostConfig' property - - "http": { - //The defaut HTTP version to being requests with (does not support http/2 yet) - "default_version": "HTTP/1.1", - //The maxium size (in bytes) of response messges that will be compressed - "compression_limit": 2048000, - //Minium response size (in bytes) to compress - "compression_minimum": 2048, - //The size of the buffer to use when parsing multipart/form data uploads - "multipart_max_buf_size": 20480, - //The maxium ammount of data (in bytes) allows for mulitpart/form data file uploads - "multipart_max_size": 80240, - //Absolute maximum size (in bytes) of the request entity body (exludes headers) - "max_entity_size": 51200, - //Keepalive ms for HTTP1.1 keepalive connections - "keepalive_ms": 1000000, - //The buffer size to use when parsing headers (also the maxium request header size allowed) - "header_buf_size": 8128, - //The maxium number of headers allowed in an HTTP request message - "max_request_header_count": 50, - //The maxium number of allowed network connections, before 503s will be issued automatically and connections closed - "max_connections": 5000, - //The size in bytes of the buffer to use when writing response messages - "response_buf_size": 65535, - //time (in ms) to wait for a response from an active connection in recv mode, before dropping it - "recv_timeout_ms": 5000, - //Time in ms to wait for the client to accept transport data before terminating the connection - "send_timeout_ms": 60000, - //The size (in bytes) of the buffer used to store all response header data - "response_header_buf_size": 16384, - //Max number of file uploads allowed per request - "max_uploads_per_request": 10 - }, - - //Path to managed compressor library - "compression_lib": "lib/vnlib.net.compression/VNLib.Net.Compression.dll", - - //Setup the native lib - "vnlib.net.compression": { - "lib_path": "lib/vnlib_compress.dll", - "level": 1 - }, - - //Maximum ammount of time a request is allowed to be processed (includes loading or waiting for sessions) before operations will be cancelled and a 503 returned - "max_execution_time_ms": 20000, - - //Collection of objects to define hosts+interfaces to build server listeners from - "virtual_hosts": [ - { - //Enable nginx-style connection tracing for this endpoint - "trace": false, - - //The interface to bind to, you may not mix TLS and non-TLS connections on the same interface - "interface": { - "address": "0.0.0.0", - "port": 8080 - }, - - //The directory path for files served by this endpoint - "path": "dist", - - //The hostname to listen for, "*" as wildcard, and "[system]" as the default hostname for the current machine - //"hostname": "*", - - //Or specify an array of hostnames instead, the hostnames array property takes priority over the single hostname property, each must be unique - "hostnames": [ "*" ], - - //Collection of "trusted" servers to allow proxy header support from - "downstream_servers": [], - - //Specify a list of ip addresses that are allowed to connect to the server, 403 will be returned if connections are not on this list - //whitelist works behind a trusted downstream server that supports X-Forwared-For headers - //"whitelist": [ "127.0.0.1" ], - - //A list of file extensions to deny access to, if a resource is requested and has one of the following extensions, a 404 is returned - "deny_extensions": [ ".env", ".yaml", ".cs" ], - - //The default file extensions to append to a resource that does not have a file extension - "default_files": [ "index.html" ], - - //Key-value headers object, some headers are special and are controlled by the vh processor - "headers": { - "X-Content-Type-Options": "nosniff", - "X-Xss-Protection": "1; mode=block", - "X-Frame-Options": "DENY", - "Server": "VNLib.Webserver", - "Content-Security-Policy": "default-src 'self' 'unsafe-inline'; img-src 'self' blob: data:; frame-src 'none'; object-src 'none'; referrer no-referrer-when-downgrade; upgrade-insecure-requests; block-all-mixed-content;" - }, - - //Enables cors support for all endpoints and header controls, if false, all endpoints that are send CORS request headers will be forbidden - //"enable_cors": true, - - //Allowed cors authoriy domains - //"cors_allowed_authority": [ "localhost:8080" ], - - //Define a TLS certificate (enables TLS on the interface) - "ssl": { - - //Cert may be pem or pfx (include private key in pfx, or include private key in a pem file) - "cert": "ssl/cert.pem", - - //A pem encoded private key, REQUIRED if using a PEM certificate, may be encrypted with a password - "privkey": "ssl/key.pem", - - //An optional password for the ssl private key - //"password": "plain-text-password", - - //requires that any client connecting to this host present a valid certificate - "client_cert_required": false - }, - - //A list of error file objects, files are loaded into memory (and watched for changes) and returned when the specified error code occurs - "error_files": [], - - //The default - "cache_default_sec": 864000 - } - ], - - - //Defines the directory where plugin's are to be loaded from - "plugins": { - //Hot-reload creates collectable assemblies that allow full re-load support in the host application, should only be used for development purposes! - "hot_reload": false, - "reload_delay_sec": 2, - "path": "plugins", - "config_dir": "config", - "assets": "plugins/assets/" - }, - - "sys_log": { - "path": "data/logs/syslog.txt", - //"template": "serilog template for writing to file", - "flush_sec": 5, - "retained_files": 10, - "file_size_limit": 10485760, - "interval": "infinite" - }, - - "app_log": { - "path": "data/logs/applog.txt", - //"template": "serilog template for writing to file", - "flush_sec": 5, - "retained_files": 10, - "file_size_limit": 10485760, - "interval": "infinite" - }, - - //Sql for the users database - "sql": { - "debug": false, - "provider": "VNLib.Plugins.Extensions.Sql.SQLite.dll", - "source": "data/simple-bookmark.db" //For sqlite only - }, - - //caching should be setup globally after VNCache #78a47dd - "cache": { - //Load the vncache dll - "assembly_name": "VNLib.Data.Caching.Providers.VNCache.dll", - //Defaulting to memory only for now - "memory_only": true, - //enable memory cache - "memory_cache": { - "buckets": 20, - "bucket_size": 5000, - "max_age_sec": 600, - "refresh_interval_sec": 60, - "zero_all": false, - "max_object_size": 8128 - } - }, - - //Global secrets object, used by the host and pluings for a specialized secrets - "secrets": { - //"db_password": "" - "passwords": "file://secrets/password-pepper.txt" - } -} diff --git a/ci/container/Dockerfile b/ci/container/Dockerfile index 33de6f1..f8cdfa3 100644 --- a/ci/container/Dockerfile +++ b/ci/container/Dockerfile @@ -42,28 +42,29 @@ WORKDIR /app #default to 8080 for TLS on TCP EXPOSE 8080/tcp -VOLUME /app/data -VOLUME /app/ssl +VOLUME /app/data \ + /app/ssl \ #expose an assets directory for custom assets install -VOLUME /app/usr/assets + /app/usr/assets #disable dotnet invariant culture on alpine ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=0 #add helper/required libraries #ENV VNLIB_SHARED_HEAP_FILE_PATH=/app/lib/libvn_rpmalloc.so not ready yet, still need to debug -ENV VNLIB_ARGON2_DLL_PATH=/app/lib/libargon2.so +ENV VNLIB_ARGON2_DLL_PATH=/app/lib/libargon2.so \ + COMPRESSION_LIB_PATH=/app/lib/libvn_compress.so #set default env variables -ENV MAX_BOOKMARKS=5000 \ - REG_TOKEN_DURATION_MIN=360 +ENV MAX_BOOKMARKS=5000 \ + MAX_CONTENT_LENGTH=5120000 \ + REG_TOKEN_DURATION_MIN=360 \ + MAX_LOGIN_ATTEMPS=10 #SQL Config ENV SQL_LIB_PATH=VNLib.Plugins.Extensions.Sql.SQLite.dll ENV SQL_CONNECTION_STRING="Data Source=data/simple-bookmark.db;" -#ACCOUNTS -ENV MAX_LOGIN_ATTEMPS=10 #HC Vault ENV HC_VAULT_ADDR="" \ diff --git a/ci/container/Taskfile.yaml b/ci/container/Taskfile.yaml index 1d52697..40c08ea 100644 --- a/ci/container/Taskfile.yaml +++ b/ci/container/Taskfile.yaml @@ -24,18 +24,12 @@ tasks: HOSTNAME: sh: echo $HOSTNAME - #build stage generates the following libraries - generates: - - "{{.USER_WORKING_DIR}}/out/libargon2.so" - - "{{.USER_WORKING_DIR}}/out/libvn_rpmalloc.so" - - "{{.USER_WORKING_DIR}}/out/libvn_compress.so" - cmds: #build argon2 native library - cd lib/argon2/ && task && cp build/libargon2.so {{.OUT_DIR}}/libargon2.so - #build rpmalloc library - - cd lib/vnlib_rpmalloc/ && task && cp build/libvn_rpmalloc.so {{.OUT_DIR}}/libvn_rpmalloc.so - #build compression + #build rpmalloc library and rewrite to a standard .dll extension + - cd lib/vnlib_rpmalloc/ && task && cp build/libvn_rpmalloc.so {{.OUT_DIR}}/libvn_rpmalloc.so + #build compression and rewrite to a standard .dll extension - cd lib/vnlib_compress/ && task && cp build/libvn_compress.so {{.OUT_DIR}}/libvn_compress.so #create a fresh self-signed cert for the container during build @@ -76,11 +70,10 @@ tasks: - powershell -Command "mkdir build, build/app, build/app/config-templates/, build/app/static/ -Force" #copy the existing linux-x64 build to the build folder, this will be the container base - powershell -Command "cp -Recurse -Force ../build/linux-x86_64/* build/app/" - #copy local scripts and config data into the build folder + #copy local scripts and raw config templates into the build folder - powershell -Command "cp -Force run.sh, Taskfile.yaml build/app/" - powershell -Command "cp -Force Dockerfile, docker-compose.yaml build/" - - powershell -Command "cp -Force static/* build/app/static/" - - powershell -Command "cp -Force config-templates/* build/app/config-templates/" + - powershell -Command "cp -Force ../config-templates/* build/app/config-templates/" prune-sql-runtimes: internal: true diff --git a/ci/container/config-templates/PageRouter-template.json b/ci/container/config-templates/PageRouter-template.json deleted file mode 100644 index 98dded3..0000000 --- a/ci/container/config-templates/PageRouter-template.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "debug": ${DEBUG_PLUGINS}, - "store": { - "route_file": "static/routes.xml" - } -}
\ No newline at end of file diff --git a/ci/container/docker-compose.yaml b/ci/container/docker-compose.yaml index fa6b0aa..4262db6 100644 --- a/ci/container/docker-compose.yaml +++ b/ci/container/docker-compose.yaml @@ -19,6 +19,8 @@ services: environment: MAX_BOOKMARKS: "5000" REG_TOKEN_DURATION_MIN: "360" #6 hours + MAX_CONTENT_LENGTH: 5120000 #5MB max upload size + MAX_LOGIN_ATTEMPS: "10" #max login attempts before user account is locked out #SQL Config SQL_LIB_PATH: "VNLib.Plugins.Extensions.Sql.SQLite.dll" @@ -36,27 +38,25 @@ services: #at least one node required if MEMCACHE_ONLY is false VNCACHE_INITIAL_NODES: "[]" - #Accounts plugin config - MAX_LOGIN_ATTEMPS: "10" - #SECRETS - #All secrets may be a raw value, read from a file, - #an environment variable, or a vault path + # All secrets may be a raw value, read from a file, + # an environment variable, or a vault path # file://mysecret.txt reads the secret from a file (case sensitive) # env://MY_SECRET reads the secret from an environment variable (case sensitive) # vault://kv/data/secret?secret=value reads the value of the mysecret key in the secret/data path + PASSWORD_PEPPER: "" #Must be a base64 encoded value, of realtivley any size - DATABASE_PASSWORD: "" - REDIS_PASSWORD: "" + DATABASE_PASSWORD: "" #overrides the 'Password' field in the SQL connection string + REDIS_PASSWORD: "" #only required if using a password protected redis server #if MEMCACHE_ONLY is false, then the following keys are required to connect to a VNCACHE cluster VNCACHE_CLIENT_PRIVATE_KEY: "" VNCACHE_CACHE_PUBLIC_KEY: "" #HTTP - HTTP_DOWNSTREAM_SERVERS: '[]' #a comma separated list of downstream ip addresses - HTTP_TRACE_ON: "false" #enable http trace logging, requires --debug CLI flag + HTTP_DOWNSTREAM_SERVERS: '[]' #a comma separated list of downstream (proxy) server ip addresses + HTTP_TRACE_ON: "false" #enable http trace logging, requires you to set --debug to SERVER_ARGS variable below - #Very Verbose plugin logging, required --debug CLI flag, prints literally everything to the logger + #Very Verbose plugin logging, required --debug CLI flag, prints literally everything to the logger (it's annoying) DEBUG_PLUGINS: "false" SERVER_ARGS: "--setup" #remove the setup flag after you are done setting up the server diff --git a/ci/container/run.sh b/ci/container/run.sh index b452f2a..d829509 100644 --- a/ci/container/run.sh +++ b/ci/container/run.sh @@ -6,6 +6,9 @@ echo "Generating configuration files" rm -rf config && mkdir config +#move the routes xml file to the output config dir +cp config-templates/routes.xml config/routes.xml + #substitude all -template files in the config-templates dir and write them to the config dir for file in config-templates/*-template.json; do envsubst < $file > config/$(basename $file -template.json).json diff --git a/ci/container/static/routes.xml b/ci/container/static/routes.xml deleted file mode 100644 index 85f9830..0000000 --- a/ci/container/static/routes.xml +++ /dev/null @@ -1,46 +0,0 @@ -<?xml version="1.0" encoding="utf-8" ?> - -<!--Routes container element holds a collection of route elements--> -<routes> - <!-- - Example route configuration for a single page app - where the tree looks like this: - / (index.html) - /assets (assets directory) (css and js files) - - Wildcard hosts match all hosts that do not have rules with more specific hosts - --> - - <!-- - Allow assets directory to pass through for all requests, using the Continue routine (1) - - Because this route has a more specific path than the catch all route - it will be processed first - --> - <route routine="1" privilege="0"> - - <!--Wildcard host--> - <hostname>*</hostname> - - <!--All paths that start with /assets/ will be matched--> - <path>/assets/*</path> - </route> - - <!--Show the index file when navigating to /--> - <route routine="4" privilege="0"> - <hostname>*</hostname> - <path>/</path> - <alternate>index.html</alternate> - </route> - - <!--Redirect all other pages back to the app (homepage)--> - <route routine="2" privilege="0"> - <hostname>*</hostname> - <path>/*</path> - <alternate>/</alternate> - </route> - - - <!--All routes that do not match will be allowed, this is only / since it does not have a matching rule--> - -</routes>
\ No newline at end of file diff --git a/ci/install.ps1 b/ci/install.ps1 index b15dfc2..3c2a2f4 100644 --- a/ci/install.ps1 +++ b/ci/install.ps1 @@ -1,7 +1,7 @@ param([String] $BaseUrl, [String] $ModuleName, [String] $ProjectName, [String]$FileName, [String]$Version) #random delays to space out the downloads -$randomDelay = Get-Random -Minimum 300 -Maximum 1000 +$randomDelay = Get-Random -Minimum 1000 -Maximum 2000 Start-Sleep -Milliseconds $randomDelay $_src = "$BaseUrl/$ModuleName/$Version/$ProjectName/$FileName" diff --git a/ci/plugins.taskfile.yaml b/ci/plugins.taskfile.yaml index efb7ca0..17b3036 100644 --- a/ci/plugins.taskfile.yaml +++ b/ci/plugins.taskfile.yaml @@ -10,6 +10,7 @@ includes: taskfile: install.taskfile.yaml vars: + PLUGIN_NAME: 'SimpleBookmark' CORE_VERSION: '1b590c2517fef110564943ed8a10edd11fa758b0' ESSENTIALS_VERSION: '451091e93b5feee7a5e01d3a81f5d63efa7ea8be' CACHE_VERSION: '930980a1e6b5db24dd4d8beeb115e4279ea79ee2' @@ -35,9 +36,9 @@ tasks: cmds: - echo "Installing and configuring plugins and UI" - - task: build-bookmarks + - task: build-proj - build-bookmarks: + build-proj: cmds: #build front-end - cd ../front-end && npm install && npm run build @@ -46,9 +47,9 @@ tasks: #build the plugin - cmd: cd ../back-end/src && task build - - powershell -Command "mkdir plugins/SimpleBookmark -Force" + - powershell -Command "mkdir plugins/{{.PLUGIN_NAME}} -Force" #copy the plugin output - - powershell -Command "cp -Recurse -Force ../back-end/src/bin/release/net8.0/publish/* plugins/SimpleBookmark/" + - powershell -Command "cp -Recurse -Force ../back-end/src/bin/release/net8.0/publish/* plugins/{{.PLUGIN_NAME}}/" install-accounts: cmds: diff --git a/ci/release.taskfile.yaml b/ci/release.taskfile.yaml index 14806bc..549ee8c 100644 --- a/ci/release.taskfile.yaml +++ b/ci/release.taskfile.yaml @@ -13,13 +13,13 @@ vars: tasks: default: - desc: "Runs the Simple-Bookmark server" + desc: "Runs the server in realease mode" interactive: true cmds: - task: run run: - desc: "Runs the Simple-Bookmark server" + desc: "Runs the server in release mode" silent: true interactive: true env: diff --git a/ci/taskfile.yaml b/ci/taskfile.yaml index ed61cf9..0221bc9 100644 --- a/ci/taskfile.yaml +++ b/ci/taskfile.yaml @@ -34,6 +34,8 @@ tasks: - cmd: powershell -Command "mkdir lib -Force" ignore_error: true + + - task: compile-config - task: plugins:all #remove runtime-shared libs before they are copied @@ -86,7 +88,7 @@ tasks: - task: pb-parallel #cleanup unnecessary build files that clog up the pipeline - - for: [ build, plugins, dist, lib, webserver ] + - for: [ build, plugins, dist, lib, webserver, config ] cmd: powershell -Command "rm -Recurse '{{.ITEM}}'" ignore_error: true @@ -119,7 +121,7 @@ tasks: ignore_error: true #copy build files for target os - - for: [ plugins, dist, lib, config, webserver ] + - for: [ plugins, dist, lib, config, webserver, config ] cmd: powershell -Command "cp -Recurse -Force {{.ITEM}} {{.BUILD_DIR}}" #copy release taskfile and rename it @@ -132,6 +134,15 @@ tasks: ignore_error: true - cd build/{{.TARGET_OS}} && tar -czf ../../bin/{{.TARGET_OS}}-release.tgz . + compile-config: + internal: false + dotenv: ['build.env'] #use the local .env file when compiling config variables + cmds: + - cmd: powershell mkdir config/ -Force + ignore_error: true + - cmd: powershell './compile.ps1' -InputDir config-templates/ -OutputDir config/ + - cmd: powershell cp config-templates/routes.xml config/routes.xml -Force + prune-plugin-libs: cmds: - for: ['vnlib.utils.dll' , 'vnlib.net.http.dll', 'VNLib.Hashing.Portable.dll', 'VNLib.Plugins.Essentials.dll', 'VNLib.Plugins.dll', 'Serilog.dll', 'Serilog.Sinks.Console.dll', 'Serilog.Sinks.File.dll'] @@ -189,7 +200,7 @@ tasks: clean: ignore_error: true cmds: - - for: [ build/, bin/, dist/, plugins/, lib/, webserver/ ] + - for: [ build/, bin/, dist/, plugins/, lib/, webserver/, config/ ] cmd: powershell -Command "rm -Recurse -Force '{{.ITEM}}'" - task: container:clean
\ No newline at end of file |