diff options
| author |  vnugent <public@vaughnnugent.com> | 2024-08-04 17:14:10 -0400 |
|---|---|---|
| committer | vnugent <public@vaughnnugent.com> | 2024-08-04 17:14:10 -0400 |
| commit | 0419f315e5689e043f311203ab8e61f69f1ee1d6 (patch) | |
| tree | f077b9d32d787bdef6e2cded203bc459a23dae7b | |
| parent | 7be5d6648e633ba46a270ca5784de6f4a5a4e0a9 (diff) | |
Squashed commit of the following:
commit bf6085a67a25c0242e3f170c3e617a08498d9ad0
Author: vnugent <public@vaughnnugent.com>
Date: Sun Aug 4 16:58:09 2024 -0400
fix compression source tree and source package
commit 4eb2cf913495e8a7c8c9ad3fceb3bff2a1b2a072
Author: vnugent <public@vaughnnugent.com>
Date: Sun Aug 4 15:49:27 2024 -0400
push to codeberg
commit 0b69bc760f87efab73ca6efb454b30a3393be269
Author: vnugent <public@vaughnnugent.com>
Date: Sun Aug 4 15:36:26 2024 -0400
consolidate log config
commit b8841c2218133bb8692e30cee0cfc719bfa5e9a0
Author: vnugent <public@vaughnnugent.com>
Date: Thu Aug 1 21:30:59 2024 -0400
update compression copyright data and add package.json
commit 08020ccace1474e27702ad6575259e799ca56b63
Merge: 904560a ff1765d
Author: vnugent <public@vaughnnugent.com>
Date: Thu Aug 1 21:26:47 2024 -0400
Merge branch 'develop' into slurp-webserver
commit ff1765d3aa4d98fd223c47d16fca8e3e13a4d894
Author: vnugent <public@vaughnnugent.com>
Date: Thu Aug 1 21:26:05 2024 -0400
set trasnport manager setters private for debug
commit 904560a7b5eafd7580fb0a03e778d1751e72a503
Author: vnugent <public@vaughnnugent.com>
Date: Thu Aug 1 21:13:04 2024 -0400
build(app): swallow vnlib.webserver into core & build updates
commit 6af95e61212611908d39235222474d4038e10fcd
Author: vnugent <public@vaughnnugent.com>
Date: Wed Jul 31 15:14:07 2024 -0400
ci: clean up heapapi header & better formalize taskfiles
commit 52965ce8bb0b06f59b07c7f6b5a9de6bfbc22b40
Author: vnugent <public@vaughnnugent.com>
Date: Mon Jul 29 12:26:32 2024 -0400
move dev-init to module level for vnbuild module init
commit e66b75642f654f623a3115bd3586d567d1554726
Merge: a4dacd2 7be5d66
Author: vnugent <public@vaughnnugent.com>
Date: Sun Jul 28 19:16:13 2024 -0400
Merge branch 'master' into develop
commit a4dacd2909426bf628c1eee1253cc5c8a01e2691
Author: vnugent <public@vaughnnugent.com>
Date: Sat Jul 27 22:41:04 2024 -0400
package updates
commit f836e09981866f5c9f2ae46990d11b186a7d12bb
Author: vnugent <public@vaughnnugent.com>
Date: Wed Jul 24 19:15:54 2024 -0400
chore: Remove argon2 docs & optional tcp resuse
commit b9b892ab2143b0ab92e4dcf0a8b043c5c6c17271
Author: vnugent <public@vaughnnugent.com>
Date: Sun Jul 21 20:57:01 2024 -0400
fix spelling Enqueue and deprecate mispelled version
commit 21ffa816f18be4b765ad740ed5d93346ec3b1fda
Author: vnugent <public@vaughnnugent.com>
Date: Sat Jul 20 19:44:31 2024 -0400
static arugment list parsing functions
commit 85cd6793818a3edd0a963bb4829a960ee6b0e022
Author: vnugent <public@vaughnnugent.com>
Date: Mon Jul 15 18:58:06 2024 -0400
chore: Just some minor checks and adjustments
commit abfb5761ee381b7e1e5342a5525ceca8c8fd81dd
Author: vnugent <public@vaughnnugent.com>
Date: Thu Jul 4 23:57:37 2024 -0400
analyzer pass
commit 4a96dbb924f2b5bf80293e4054f221efe67151dd
Author: vnugent <public@vaughnnugent.com>
Date: Thu Jul 4 22:45:28 2024 -0400
package updates
commit 38ad7d923fa8d9e463d4aaa8e35f021086a03f2d
Author: vnugent <public@vaughnnugent.com>
Date: Thu Jul 4 16:20:48 2024 -0400
mimalloc merge upstream upgrades
commit 981ba286e4793de95bf65e6588313411344c4d53
Author: vnugent <public@vaughnnugent.com>
Date: Thu Jul 4 16:04:03 2024 -0400
refactor: Refactor extensions with perf updates
commit 6b8c67888731f7dd210acdb2b1160cdbdbe30d47
Author: vnugent <public@vaughnnugent.com>
Date: Fri Jun 28 15:48:22 2024 -0400
refactor: Update service stack to reflect new loading patterns
commit 12391e9a207b60b41a074600fc2373ad3eb1c3ab
Author: vnugent <public@vaughnnugent.com>
Date: Wed Jun 26 21:01:15 2024 -0400
feat(server): Server arch update, Memory struct access
commit 92e182ceaf843f8d859d38faa8b2c0ff53207ff6
Author: vnugent <public@vaughnnugent.com>
Date: Fri Jun 21 16:02:34 2024 -0400
feat: Multi transport listeners
commit ee3620b8168a42c8e571e853c751ad5999a9b907
Author: vnugent <public@vaughnnugent.com>
Date: Tue Jun 18 21:17:28 2024 -0400
feat: Add file path caching support
commit ff0926be56fc6eafdce36411847d73bf4ce9f183
Author: vnugent <public@vaughnnugent.com>
Date: Sun Jun 16 13:08:31 2024 -0400
feat: Allow multiple plugin loading directories
commit 07ddf6738d32127926d07b1366e56d2a2308b53b
Author: vnugent <public@vaughnnugent.com>
Date: Sun Jun 16 01:12:07 2024 -0400
perf: Absolutely yuge perf boosts
commit ff15c05a9c3e632c39f3889820fb7d889342b452
Author: vnugent <public@vaughnnugent.com>
Date: Fri Jun 14 14:16:24 2024 -0400
fix: Improper request buffer property assignment
commit 7d2987f1d4048c30808a85798e32c99747f6cfe3
Author: vnugent <public@vaughnnugent.com>
Date: Thu Jun 13 21:57:34 2024 -0400
perf: Async pre-buffer to avoid sync buffer
commit 75c1d0cbf9a5a7856c544671a45f1b4312ffe7ce
Author: vnugent <public@vaughnnugent.com>
Date: Tue Jun 11 22:11:45 2024 -0400
feat: Add a default site adapater and interceptors
commit a7c739b7db9a17622cee751fe0e8a10e4b84b48b
Author: vnugent <public@vaughnnugent.com>
Date: Sun Jun 9 13:05:12 2024 -0400
chore: Package updated
commit b4b506a4b6c7c1e90b5b0980e4cfe0460e7546a2
Author: vnugent <public@vaughnnugent.com>
Date: Sat Jun 8 21:54:52 2024 -0400
some minor touchups
commit 2160510fcc22a8574b0090fd91ca29072f45ab59
Author: vnugent <public@vaughnnugent.com>
Date: Fri May 31 15:12:20 2024 -0400
refactor: Immutable tcp listeners
commit 51cb4eb93e4f1b4c47d35b105e72af1fe771abcc
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 30 17:31:16 2024 -0400
refactor: minor non-breaking changes to VNEncoding
commit 768ddc1eb949266d693f96c67d734e881bd59374
Merge: 9a835fe 1b590c2
Author: vnugent <public@vaughnnugent.com>
Date: Wed May 22 17:50:57 2024 -0400
Merge branch 'main' into develop
commit 9a835fe12c9586ab8dd44d7c96fef4a2d6017e4b
Author: vnugent <public@vaughnnugent.com>
Date: Fri May 17 18:27:03 2024 -0400
chore: Update mimmaloc v2.1.6, update fPIC & cleanup
commit 3b7004b88acfc7f7baa3a8857a5a2f7cf3dd560e
Author: vnugent <public@vaughnnugent.com>
Date: Fri May 17 16:03:28 2024 -0400
feat: Added ReadFileDataAsync function
commit 9a964795757bf0da4dd7fcab15ad304f4ea3fdf1
Author: vnugent <public@vaughnnugent.com>
Date: Wed May 15 21:57:39 2024 -0400
refactor: Harden some argon2 password hashing
commit 4035c838c1508af0aa7e767a97431a692958ce1c
Author: vnugent <public@vaughnnugent.com>
Date: Sun May 12 16:55:32 2024 -0400
perf: Utils + http perf mods
commit f4f0d4f74250257991c57bfae74c4852c7e1ae46
Author: vnugent <public@vaughnnugent.com>
Date: Thu May 2 15:22:53 2024 -0400
feat: Buff middleware handlers
|
| Added implicit support for middleware post processing of files before the filehandler closes the connection. Also cleaned up some project file stuff
commit f0b7dca107659df1d7d4631fdbd2aae9d716d053
Merge: 8c4a45e 107b058
Author: vnugent <public@vaughnnugent.com>
Date: Sat Apr 20 12:24:05 2024 -0400
Merge branch 'main' into develop
commit 8c4a45e384accf92b1b6d748530e8d46f7de40d6
Author: vnugent <public@vaughnnugent.com>
Date: Sat Apr 20 11:10:30 2024 -0400
refactor: Overhaul C libraries and fix builds
commit 42ff77080d10b0fc9fecbbc46141e8e23a1d066a
Author: vnugent <public@vaughnnugent.com>
Date: Sat Apr 20 00:45:57 2024 -0400
fix!: Middlware array, multiple cookie set, and cookie check
commit 97e82b9d66f387f9e6d21d88ddc7a8ab8693149c
Merge: 4ca5791 e07537a
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 2 13:34:22 2024 -0400
Merge branch 'main' into develop
commit 4ca5791ed67b9834bdbd010206b30373e4705e9b
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 2 13:32:12 2024 -0400
fix: Missed ! on null pointer check
commit 9b4036377c52200c6488c98180d69a0e63321f97
Author: vnugent <public@vaughnnugent.com>
Date: Tue Apr 2 13:22:29 2024 -0400
fix: Fix _In_ macro for compression public api
commit 53a7b4b5c5b67b4a4e06e1d9098cac4bcd6afd7c
Merge: 448a93b 21130c8
Author: vnugent <public@vaughnnugent.com>
Date: Sun Mar 31 17:01:15 2024 -0400
Merge branch 'main' into develop
commit 448a93bb1d18d032087afe2476ffccb98634a54c
Author: vnugent <public@vaughnnugent.com>
Date: Sun Mar 31 16:56:51 2024 -0400
ci: fix third-party dir cleanup
commit 9afed1427472da1ea13079f98dbe27339e55ee7d
Author: vnugent <public@vaughnnugent.com>
Date: Sun Mar 31 16:43:15 2024 -0400
perf: Deprecate unsafememoryhandle span extensions
commit 3ff90da4f02af47ea6d233fdd4445337ebe36452
Author: vnugent <public@vaughnnugent.com>
Date: Sat Mar 30 21:36:18 2024 -0400
refactor: Updates, advanced tracing, http optimizations
commit 8d6b79b5ae309b36f265ba81529bcef8bfcd7414
Merge: 6c1667b 5585915
Author: vnugent <public@vaughnnugent.com>
Date: Sun Mar 24 21:01:31 2024 -0400
Merge branch 'main' into develop
commit 6c1667be23597513537f8190e2f55d65eb9b7c7a
Author: vnugent <public@vaughnnugent.com>
Date: Fri Mar 22 12:01:53 2024 -0400
refactor: Overhauled native library loading and lazy init
commit ebf688f2f974295beabf7b5def7e6f6f150551d0
Author: vnugent <public@vaughnnugent.com>
Date: Wed Mar 20 22:16:17 2024 -0400
refactor: Update compression header files and macros + Ci build
commit 9c7b564911080ccd5cbbb9851a0757b05e1e9047
Author: vnugent <public@vaughnnugent.com>
Date: Tue Mar 19 21:54:49 2024 -0400
refactor: JWK overhaul & add length getter to FileUpload
commit 6d8c3444e09561e5957491b3cc1ae858e0abdd14
Author: vnugent <public@vaughnnugent.com>
Date: Mon Mar 18 16:13:20 2024 -0400
feat: Add FNV1a software checksum and basic correction tests
commit 00d182088cecefc08ca80b1faee9bed3f215f40b
Author: vnugent <public@vaughnnugent.com>
Date: Fri Mar 15 01:05:27 2024 -0400
chore: #6 Use utils filewatcher instead of built-in
commit d513c10d9895c6693519ef1d459c6a5a76929436
Author: vnugent <public@vaughnnugent.com>
Date: Sun Mar 10 21:58:14 2024 -0400
source tree project location updated
101 files changed, 6471 insertions, 390 deletions
diff --git a/.onedev-buildspec.yml b/.onedev-buildspec.yml index 44ee930..9700986 100644 --- a/.onedev-buildspec.yml +++ b/.onedev-buildspec.yml @@ -1,14 +1,20 @@ -version: 17 +version: 33 jobs: -- name: Push to GitHub +- name: Repo Sync Push steps: - !PushRepository - name: GitHub Sync + name: Github sync push remoteUrl: https://github.com/VnUgE/VNLib.Core.git userName: VnUgE passwordSecret: git-access-token - withLfs: false - force: false + force: true + condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL + - !PushRepository + name: Codeberg push sync + remoteUrl: https://codeberg.org/VnUgE/VNLib.Core.git + userName: VnUgE + passwordSecret: codeberg-access-token + force: true condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL triggers: - !TagCreateTrigger @@ -18,13 +24,11 @@ jobs: retryCondition: never maxRetries: 3 retryDelay: 30 - cpuRequirement: 250 - memoryRequirement: 256 timeout: 3600 -- name: Pull from GitHub +- name: Repo Sync Pull steps: - !PullRepository - name: Sync from GitHub + name: GitHub sync pull remoteUrl: https://github.com/VnUgE/VNLib.Core.git userName: VnUgE passwordSecret: git-access-token @@ -39,6 +43,4 @@ jobs: retryCondition: never maxRetries: 3 retryDelay: 30 - cpuRequirement: 250 - memoryRequirement: 256 timeout: 3600 diff --git a/Module.Taskfile.yaml b/Module.Taskfile.yaml index 8d81b0e..aa5893f 100644 --- a/Module.Taskfile.yaml +++ b/Module.Taskfile.yaml @@ -11,9 +11,9 @@ version: '3' vars: - INT_DIR: '{{.SCRATCH_DIR}}/obj/{{.MODULE_NAME}}/' + INT_DIR: '{{ .SCRATCH_DIR }}/obj/{{ .MODULE_NAME }}/' MS_ARGS: '/p:RunAnalyzersDuringBuild=false /p:IntermediateOutputPath="{{.INT_DIR}}" /p:UseCommonOutputDirectory=true /p:BuildInParallel=true /p:MultiProcessorCompilation=true /p:ErrorOnDuplicatePublishOutputFiles=false' - PACK_OUT: '{{.OUTPUT_DIR}}/{{.HEAD_SHA}}/pkg' + PACK_OUT: '{{ .OUTPUT_DIR }}/{{ .HEAD_SHA }}/pkg' tasks: @@ -22,7 +22,7 @@ tasks: cmds: - git reset --hard #clean up any local changes - git remote update - - git pull origin {{.BRANCH_NAME}} --verify-signatures + - git pull origin {{ .BRANCH_NAME }} --verify-signatures #re-write semver after hard reset - dotnet-gitversion.exe /updateprojectfiles @@ -49,12 +49,24 @@ tasks: #called by build pipeline to clean module clean: + ignore_error: true cmds: #clean solution - - dotnet clean /p:BuildInParallel=true /p:MultiProcessorCompilation=true - - cmd: powershell -Command "rm {{ .ARCHIVE_FILE_NAME }} --Force" - ignore_error: true + - dotnet clean /p:BuildInParallel=true /p:MultiProcessorCompilation=true + - cmd: powershell -Command "rm {{ .ARCHIVE_FILE_NAME }} --Force" + dev-init: + desc: 'Configures the module for local development' + cmds: + #build native libs for development + - cd lib/Utils.Memory/vnlib_rpmalloc && task dev-init + - cd lib/Utils.Memory/vnlib_mimalloc && task dev-init + - cd lib/Utils.Cryptography/monocypher && task build + - cd lib/Utils.Cryptography/argon2 && task build + - cd lib/Net.Compression/vnlib_compress && task build + - cmd: echo "dev init complete" + silent: true + #Internal tasks build_debug: @@ -1,31 +1,51 @@ + # VNLib.Core -*A Mono-repo for "core" VNLib projects for simplicity.* - -### What is VNLib.Core? +<h4 align="left"> + <a href="https://github.com/VnUgE/vnlib.core"> + <img src="https://img.shields.io/github/repo-size/vnuge/vnlib.core" alt="repo-size" /> + </a> + <a href="https://www.vaughnnugent.com/Resources/Software/Modules/VNLib.Core-issues"> + <img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fwww.vaughnnugent.com%2Fapi%2Fgit%2Fissues%3Fmodule%3DVNLib.Core&query=%24%5B'result'%5D.length&label=all%20issues" alt="Issues"/> + </a> + <a href="https://github.com/VnUgE/vnlib.core/commits"> + <img src="https://img.shields.io/github/last-commit/vnuge/vnlib.core/develop" alt="Latest commit"/> + </a> + <a href="https://www.vaughnnugent.com/Resources/Software/Modules/VNLib.Core"> + <img src="https://img.shields.io/website?url=https%3A%2F%2Fwww.vaughnnugent.com" alt="Website Status"/> + </a> +</h4> + +*A mono-repo for "core" VNLib libraries and applications.* + +</br> +<h3 align="center"> + <a href="https://www.vaughnnugent.com/resources/software/modules/VNLib.Core"> + 🏠 Project Home + </a> + <a href="https://www.vaughnnugent.com/resources/software/articles?tags=docs,_VNLib.Core"> + 📖 Documentation + </a> + <a href="https://www.vaughnnugent.com/resources/software/modules"> + 📦 NuGet Feed + </a> +</h3> +</br> + +## What is VNLib.Core? VNLib is a collection of cross platform .NET/C#/C libraries that I maintain for many private and public projects. VNLib.Core is a subset of the larger collection of projects I have named VNLib. This repo contains libraries I consider to be utility only, or building blocks that are individually useful for other projects. You will likely see many or most of them used across may other VNLib type projects. These libraries are meant to be stand-alone, meaning that there are no required* external dependencies (except the [mscorelib](https://github.com/dotnet/runtime)). For example the [VNLib.Utils](lib/Utils/#) library is a standalone, 0 required dependency library that is useful for, logging, common extensions, and a significant collection of memory related utilities. -### Dependencies +## Dependencies Any libraries in this repository that contain external dependencies will be mentioned explicitly in the library's readme. I intend to limit this behavior, as it is the reason this repository exists. -### Licensing +## Licensing Projects contained in this repository are individually licensed, usually GNU copy-left but some are more open, or are modified and/or vendored projects. Builds will contain the required license txt files (along with third-party licenses) in the archive. -### Documentation -Docs and articles will be available from the docs link below . There are docs per project (as they are added) and docs tagged for the VNLib.Core module. I will also be adding "spec" or specification articles to explain the higher level concepts behind the design. - -### Contact info -Again, go to my website below, my email address is available, go ahead and send me a message. Or use the email address from my profile to send me an email - -### Links -[Home Page](https://www.vaughnnugent.com) - Website home page -[Documentation](https://www.vaughnnugent.com/resources/software/articles?tags=docs,_VNLib.Core) - Docs and articles for this module -[Builds for VNLib.Core](https://www.vaughnnugent.com/resources/software/modules/VNLib.Core) - Per-project build artifacts -[Links for Nuget Feeds](https://www.vaughnnugent.com/resources/software/modules) - Get my NuGet feed links - ## Index/NameSpaces **VNLib.** -- [Utils](lib/Utils/#) - A mutli-use library focused on reducing complexity for working with native resources, memory, asynchronous patterns and data-structures, object/resource pooling, critical resource handling, and common logging abstractions. +- [Utils](lib/Utils/#) - A mutli-use library focused on reducing complexity for working with native resources, memory, asynchronous patterns and data-structures, object/resource pooling, critical resource handling, and common logging abstractions. +- [Utils.Memory](lib/Utils.Memory/#) - Utilty libraries for native memory management framework for VNLib, includes mimalloc and rpmalloc forks. +- [Utils.Cryptography](lib/Utils.Cryptography/#) - Contains vendored copies of recommended/referenced libraries and wrapper libraries such as Argon2 and Monocypher. - [Hashing.Portable](lib/Hashing.Portable/#) - Common cryptographic/hashing relate operations in a single package with room to grow. Also Argon2 bindings - [Net.Http](lib/Net.Http/#) - High performance ^HTTP/1.1 application processing library, transport not included! For web or custom use, custom app layer protocols supported - roll your own web-sockets or SSE if you want! (See [Essentials](lib/Plugins.Essentials/#) library if you want to use mine! It's not bad.) - [Net.Transport.SimpleTCP](lib/Net.Transport.SimpleTCP/#) - A not-so-simple, performance oriented, low/no allocation, .NET/C# native, TCP listener library using the System.IO.Pipelines architecture with SSL/TLS support. @@ -35,38 +55,18 @@ Again, go to my website below, my email address is available, go ahead and send - [Plugins.Essentials.ServiceStack](lib/Plugins.Essentials.ServiceStack/#) - A library for scaffolding structured web applications from the individual http and supporting libraries into a completely managed http service stack for an entire application. - [Plugins.PluginBase](lib/Plugins.PluginBase/#) - Base library/api for plugin developers to build fully managed/supported runtime loaded plugins, without worrying about the plumbing, such as the IPlugin api, endpoint creation, and logging! This library is required if you wish to use most of the Plugin.Extensions libraries. - [Net.Messaging.FBM](lib/Net.Messaging.FBM/#) - Fixed Buffer Messaging protocol, high performance, request/response architecture, client & server library, built atop http and web-sockets. As implied, relies on fixed sized internal buffers that are negotiated to transfer data with minimal overhead for known messaging architectures. -- [Utils.Memory](lib/Utils.Memory/#) - Utilty libraries for native memory management framework for VNLib, includes mimalloc and rpmalloc forks. - [Net.Compression](lib/Net.Compression/#) - A cross platform native compression provider and IHttpCompressorManager configured for runtime dynamic loading for high performance native response data compression. - [Net.Rest.Client](lib/Net.Rest.Client/#) - A library for defining REST api clients via a fluent api by defining sites and endpoints, OAuth2 authenticator for RestSharp, and a simple RestSharp client pool. -- [Utils.Cryptography](lib/Utils.Cryptography/#) - Contains vendored copies of recommended/referenced libraries and wrapper libraries such as Argon2 and Monocypher. +- [WebServer](apps/VNLib.WebServer/#) - A high performance, reference .NET 8 web server built the Essentials web framework for building fast, plugin-driven web/http services -The **third-party** directory contains third-party librarires that are forked, modified, and vendored for use in VNLib projects that I will activley maintain. - -## Builds & Source -Builds contain the individual components listed below packaged per-project, available for download on my website. Build packages will be tgz archives (except for nuget packages). You can obtain debug and release builds, along with per-project source code - -[Link to builds](https://www.vaughnnugent.com/resources/software/modules/VNLib.Core) - -### NuGet Feeds -NuGet feeds for release and debug packages are available on my website as the links are subject to change. Debug packages contain symbols and source code, and both contain XML documentation. (Search is now working) - -[Links for Nuget Feeds](https://www.vaughnnugent.com/resources/software/modules) +The **third-party** directory contains third-party libraries that are forked, modified, and vendored for use in VNLib projects that I will actively maintain. ## Branches -There are currently two branches I use, master and develop. Develop is the my default building branch, all changes are merged into master when I am satisfied. An internal PR is opened, reviewed and merged into master, you will notice this merge often consists of multiple commits. The develop branch will ALWAYS be ahead of the master branch. +There are currently two branches I use, master and develop. Develop is the my default building branch, all changes are merged into master when I am satisfied. An internal PR is opened, reviewed and merged into master using squash commits to preserve master history and development speed. The develop branch will ALWAYS be ahead of the master branch. -## Notes -As with all of my VNlib projects, there are in a very early pre-release state. This is my first large-scale software engineering project and its not in highest of priorities to be a good "software developer", sorry :smiley:. I care about the highest performance code I know how to make for my applications regardless of complexity. I use ALL of the packages/libraries in these projects in my own infrastructure and as issues appear I try to fix and test them before publishing the build packages on my website (see builds below). I understand that most of my projects are often re-inventing the wheel, I'm aware, I enjoy boilerplate because it's mine... This all subject to change without warning. - -### How is the code for this repo maintained? +## How is the repo maintained? I use many internal tools to build and maintain these projects. I use [OneDev](https://code.onedev.io/) for my internal source control (as well as other git tools) and code updates are pushed to GitHub as part of a build process. I use [Task](https://taskfile.dev) in the build process to publish builds for all projects in this repository. Builds are publicly available on my website from the link below. I do *not* intend to expose my internal tools for security reasons. I prefer to keep my dependencies/processes internal, and will be relying on GitHub for as little as possible. -### My code style -It changes when I learn new things, it will continue to evolve. I live on hard-mode, aka ADHD (surprise...), so I prefer verbosity and very heavy comments, recognition is better than recall. Comments may get outdated from time to time, sorry, again pattern recognition helps me remember what all 450k lines of code do at a glance (among all my repos at this stage). - -### Why so many individual projects? - Motivation (TLDR) -I originally wanted the ability to listen for simple HTTP connections without the beast of ASP and learning curve involved, and wanted to test my skills. I wanted something that was up-to-date, actually cross-platform, and maybe even compete with ASP. At the same time, I wanted to provide abstractions and layers for other engineers to be able to choose the pieces they want to their project. Finally, I also wanted few to no 3rd party dependencies. These libraries were eventually broken down into their useful pieces and may continue to expand in the future. - ## Contributing I'm not currently looking for code contributors for this project and others, and NO contributions will be accepted through GitHub directly. I would prefer you contact me via my contact information on my website if possible. diff --git a/Taskfile.yaml b/Taskfile.yaml index 0d73872..ad0e837 100644 --- a/Taskfile.yaml +++ b/Taskfile.yaml @@ -10,65 +10,53 @@ version: '3' vars: - TARGET: '{{.USER_WORKING_DIR}}/bin' + BINARY_DIR: 'bin' RELEASE_DIR: "./bin/release/{{.TARGET_FRAMEWORK}}/publish" + TARGET_DIR: '{{ .USER_WORKING_DIR }}/{{ .BINARY_DIR }}' tasks: - dev-setup: - desc: 'Sets up all native libraries for local development' - cmds: - #build native libs for development - - cd lib/Utils.Memory/vnlib_rpmalloc && task build - - cd lib/Utils.Memory/vnlib_mimalloc && task build - - cd lib/Utils.Cryptography/monocypher && task build - - cd lib/Utils.Cryptography/argon2 && task build - - cd lib/Net.Compression/vnlib_compress && task build - - echo "dev setup complete" - #when build succeeds, archive the output into a tgz postbuild_success: dir: '{{.USER_WORKING_DIR}}' cmds: - #pack up source code - - task: packsource - - #run post in debug mode - - task: postbuild - vars: { BUILD_MODE: debug } - #remove uncessary files from the release dir - powershell -Command "Get-ChildItem -Recurse '{{.RELEASE_DIR}}/' -Include *.pdb,*.xml | Remove-Item" - #run post in release mode - - task: postbuild - vars: { BUILD_MODE: release } - - - postbuild_failed: - dir: '{{.USER_WORKING_DIR}}' - cmds: [] + - task: post_parallel + post_parallel: + internal: true + deps: + - task: packsource + - task: postbuild + vars: { BUILD_MODE: debug } + - task: postbuild + vars: { BUILD_MODE: release } postbuild: - dir: '{{.USER_WORKING_DIR}}' + dir: '{{.USER_WORKING_DIR}}' internal: true vars: #the build output directory - BUILD_OUT: "{{.USER_WORKING_DIR}}/bin/{{.BUILD_MODE}}/{{.TARGET_FRAMEWORK}}/publish" + BUILD_OUT: "{{ .BINARY_DIR }}/{{ .BUILD_MODE }}/{{ .TARGET_FRAMEWORK }}/publish" cmds: - - cd .. && powershell -Command "Copy-Item -Path ./build.readme.txt -Destination '{{.BUILD_OUT}}/readme.txt'" + - powershell cp ../build.readme.txt '{{.BUILD_OUT}}/readme.txt' #tar outputs - - cd "{{.BUILD_OUT}}" && tar -czf "{{.TARGET}}/{{.BUILD_MODE}}.tgz" . + - cd "{{ .BUILD_OUT }}" && tar -czf "{{ .TARGET_DIR }}/{{ .BUILD_MODE }}.tgz" . packsource: dir: '{{.USER_WORKING_DIR}}' internal: true cmds: #copy source code to target - - powershell -Command "Get-ChildItem -Include *.cs,*.csproj -Recurse | Where { \$_.FullName -notlike '*\obj\*' -and \$_.FullName -notlike '*\bin\*' } | Resolve-Path -Relative | tar --files-from - -czf '{{.TARGET}}/src.tgz'" + - + powershell -Command "Get-ChildItem -Include *.cs,*.csproj -Recurse + | Where { \$_.FullName -notlike '*\obj\*' -and \$_.FullName -notlike '*\bin\*' } + | Resolve-Path -Relative + | tar --files-from - -czf '{{ .TARGET_DIR }}/src.tgz'" #Remove the output dirs on clean diff --git a/apps/VNLib.WebServer/LICENSE b/apps/VNLib.WebServer/LICENSE new file mode 100644 index 0000000..1a24a8e --- /dev/null +++ b/apps/VNLib.WebServer/LICENSE @@ -0,0 +1,195 @@ +Copyright (c) 2024 Vaughn Nugent + +Contact information + Name: Vaughn Nugent + Email: vnpublic[at]proton.me + Website: https://www.vaughnnugent.com + +The software in this repository is licensed under the GNU Affero GPL version 3.0 (or any later version). + +GNU AFFERO GENERAL PUBLIC LICENSE + +Version 3, 19 November 2007 + +Copyright © 2007 Free Software Foundation, Inc. <https://fsf.org/> +Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. +Preamble + +The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software. + +The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. + +When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. + +Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software. + +A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public. + +The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version. + +An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license. + +The precise terms and conditions for copying, distribution and modification follow. +TERMS AND CONDITIONS +0. Definitions. + +"This License" refers to version 3 of the GNU Affero General Public License. + +"Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. + +"The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. + +To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. + +A "covered work" means either the unmodified Program or a work based on the Program. + +To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. + +To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. + +An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. +1. Source Code. + +The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. + +A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. + +The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. + +The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. + +The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. + +The Corresponding Source for a work in source code form is that same work. +2. Basic Permissions. + +All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. + +You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. + +Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. +3. Protecting Users' Legal Rights From Anti-Circumvention Law. + +No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. + +When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. +4. Conveying Verbatim Copies. + +You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. + +You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. +5. Conveying Modified Source Versions. + +You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified it, and giving a relevant date. + b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". + c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. + d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. + +A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. +6. Conveying Non-Source Forms. + +You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: + + a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. + b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. + c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. + d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. + e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. + +A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. + +A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. + +"Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. + +If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). + +The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. + +Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. +7. Additional Terms. + +"Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. + +When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. + +Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or + b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or + c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or + d) Limiting the use for publicity purposes of names of licensors or authors of the material; or + e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or + f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. + +All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. + +If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. + +Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. +8. Termination. + +You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). + +However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. + +Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. + +Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. +9. Acceptance Not Required for Having Copies. + +You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. +10. Automatic Licensing of Downstream Recipients. + +Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. + +An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. + +You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. +11. Patents. + +A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". + +A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. + +Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. + +In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. + +If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. + +If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. + +A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. + +Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. +12. No Surrender of Others' Freedom. + +If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. +13. Remote Network Interaction; Use with the GNU General Public License. + +Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. + +Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License. +14. Revised Versions of this License. + +The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation. + +If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. + +Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. +15. Disclaimer of Warranty. + +THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. +16. Limitation of Liability. + +IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. +17. Interpretation of Sections 15 and 16. + +If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. + +END OF TERMS AND CONDITIONS
\ No newline at end of file diff --git a/apps/VNLib.WebServer/README.md b/apps/VNLib.WebServer/README.md new file mode 100644 index 0000000..06b8199 --- /dev/null +++ b/apps/VNLib.WebServer/README.md @@ -0,0 +1,38 @@ +<h1 align="center">VNLib.WebServer</h1> + +<p align="center"> +A high performance, reference .NET 8 web server built with VNLib.Core and Essentials web framework for building fast, plugin-driven web/http services. +</p> + +<h4 align="center"> + <a href="https://www.vaughnnugent.com/resources/software/modules/VNLib.Core?p=vnlib.webserver">Builds</a> | + <a href="https://www.vaughnnugent.com/resources/software">My Software</a> | + <a href="https://www.vaughnnugent.com/resources/software/articles?tags=_vnlib.webserver">Documentation</a> +</h4> + +<img src="https://www.vaughnnugent.com/public/blogs/docs/content/pvsa5sttunurphjljc73jhg3mi.png" width="100%"> + +## Short Intro +VNLib.Webserver is a runtime host for http or "web" bassed server applications. As a standalone application, it can only do basic http file processing via virtual hosts similar to nginx and appache, but when plugins are configured, it becomes a highly versitlile dynamic server application. VNLib.WebServer inclues only the bare minimum binaries for any application, and is infinitly expandable using dynamic assembly loading. + +### Some features +- HTTP 0.9-1.1 support with granular control over http and tcp settings +- Virtual Hosts: many-to-many hostname-transport configuration (similar to nginx) +- Strong TLS support using .NET SslStream library +- Static file processing +- JSON and Yaml coniguration language support +- CORS resource support and protections +- Per-host error file caching (ex: 403, 404) +- IP based whitelist and blacklist +- File protection by extension +- File path probing/autocomplete (ex: / -> /index.html or / -> custom_file.ext) +- Primitive file http cache control +- Optional console command listener for issuing control commands +- HTTP compression with external library and Brotli as a built-in fallback +- HTTP range support for files + +## License +The software in this repository is licensed under the GNU Affero General Public License (or any later version). See the LICENSE files for more information. + +## Donations +If you like this project and want to support it or motivate me for faster development, please see the parent project's [README](../../#). diff --git a/apps/VNLib.WebServer/Taskfile.yaml b/apps/VNLib.WebServer/Taskfile.yaml new file mode 100644 index 0000000..1f0d71b --- /dev/null +++ b/apps/VNLib.WebServer/Taskfile.yaml @@ -0,0 +1,112 @@ +# https://taskfile.dev + +version: '3' + +vars: + BINARY_DIR: 'bin' + INT_DIR: '{{ .SCRATCH_DIR }}/obj/{{ .MODULE_NAME }}/' + MS_ARGS: '/p:RunAnalyzersDuringBuild=false /p:IntermediateOutputPath="{{ .INT_DIR }}" /p:UseCommonOutputDirectory=true /p:BuildInParallel=true /p:MultiProcessorCompilation=true /p:ErrorOnDuplicatePublishOutputFiles=false' + +tasks: + + #called by build pipeline to build module + build: + dir: '{{.USER_WORKING_DIR}}' + cmds: + + #managed source supports arm32 unmanaged does not + - for: [ win-x64, linux-x64, osx-x64, linux-arm64, linux-arm ] + cmd: powershell -Command 'dotnet publish -c debug -r {{ .ITEM }} {{ .BUILD_FLAGS }} --sc false {{ .MS_ARGS }}' + + #build release mode after all debug builds + - for: [ win-x64, linux-x64, osx-x64, linux-arm64, linux-arm ] + cmd: powershell -Command 'dotnet publish -c release -r {{ .ITEM }} {{ .BUILD_FLAGS }} --sc false {{ .MS_ARGS }}' + + #package as a tool + #- dotnet pack -c debug {{.MS_ARGS}} -o "{{.PACK_OUT}}/debug/" -p:PackageVersion={{.BUILD_VERSION}} + + #when build succeeds, archive the output into a tgz + postbuild_success: + dir: '{{ .USER_WORKING_DIR }}' + vars: + RELEASE_DIR: "{{ .BINARY_DIR }}/release/{{ .TARGET_FRAMEWORK }}" + + cmds: + + - cmd: powershell mkdir {{ .BINARY_DIR }} -Force + + #remove uncessary files from the release dir + - cmd: powershell -Command "Get-ChildItem -Recurse '{{ .RELEASE_DIR }}/' -Include *.pdb,*.xml | Remove-Item" + + - task: pack_parallel + + pack_parallel: + internal: true + deps: + - task: pack_source + + - task: postbuild + vars: { BUILD_MODE: 'debug', TARGET_OS: linux-x64 } + - task: postbuild + vars: { BUILD_MODE: 'debug', TARGET_OS: linux-arm } + - task: postbuild + vars: { BUILD_MODE: 'debug', TARGET_OS: linux-arm64 } + - task: postbuild + vars: { BUILD_MODE: 'debug', TARGET_OS: win-x64 } + - task: postbuild + vars: { BUILD_MODE: 'debug', TARGET_OS: osx-x64 } + + - task: postbuild + vars: { BUILD_MODE: 'release', TARGET_OS: linux-x64 } + - task: postbuild + vars: { BUILD_MODE: 'release', TARGET_OS: linux-arm } + - task: postbuild + vars: { BUILD_MODE: 'release', TARGET_OS: linux-arm64 } + - task: postbuild + vars: { BUILD_MODE: 'release', TARGET_OS: win-x64 } + - task: postbuild + vars: { BUILD_MODE: 'release', TARGET_OS: osx-x64 } + + postbuild: + internal: true + dir: '{{ .USER_WORKING_DIR }}' + vars: + BUILD_DIR: "{{ .BINARY_DIR }}/{{ .BUILD_MODE }}/{{ .TARGET_FRAMEWORK }}/{{ .TARGET_OS }}/publish" + cmds: + #copy and readme to target + - powershell cp ../build.readme.txt '{{ .BUILD_DIR }}/readme.txt' + + #copy release taskfile + #- cd .. && powershell -Command "Copy-Item -Path ./release.taskfile.yaml -Destination '{{.BUILD_DIR}}/Taskfile.yaml'" + + #tar outputs + - cd "{{ .BUILD_DIR }}" && tar -czf "{{ .USER_WORKING_DIR }}/{{ .BINARY_DIR }}/{{ .TARGET_OS }}-{{ .BUILD_MODE }}.tgz" . + + pack_source: + internal: true + dir: '{{ .USER_WORKING_DIR }}' + vars: + EXCLUDES: + --exclude='src/bin/*' + --exclude='src/obj/*' + #--exclude='.tarignore' + INCLUDES: + src/* + LICENSE + Taskfile.yaml + README.md + + cmds: + #pack up source code and put in output + - cmd: cd .. && tar {{ .EXCLUDES }} -czf '{{ .USER_WORKING_DIR }}/{{ .BINARY_DIR }}/src.tgz' {{ .INCLUDES }} + + #Remove the output dirs on clean + clean: + dir: '{{.USER_WORKING_DIR}}' + ignore_error: true + cmds: + - dotnet clean /p:BuildInParallel=true /p:MultiProcessorCompilation=true + + - for: [ bin/, obj/ ] + cmd: powershell Remove-Item -Recurse -Force '{{.USER_WORKING_DIR}}/{{.ITEM}}' + diff --git a/apps/VNLib.WebServer/build.readme.txt b/apps/VNLib.WebServer/build.readme.txt new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/apps/VNLib.WebServer/build.readme.txt diff --git a/apps/VNLib.WebServer/release.taskfile.yaml b/apps/VNLib.WebServer/release.taskfile.yaml new file mode 100644 index 0000000..86c2538 --- /dev/null +++ b/apps/VNLib.WebServer/release.taskfile.yaml @@ -0,0 +1,56 @@ +# https://taskfile.dev + +version: '3' + +vars: + + +tasks: + + default: + desc: "Runs the server in a default startup mode for production use" + vars: + ARGS: "--config ../config/config.json --input-off" + + cmds: + - task: run + vars: { ARGS: '{{.ARGS}}' } + + run-debug: + env: + VNLIB_SHARED_HEAP_DIAGNOSTICS: 1 + MIMALLOC_VERBOSE: 1 #incase mimalloc is used we can force debugging + + vars: + ARGS: "--config ../config/config.json --debug" + + cmds: + - task: run + vars: { ARGS: '{{.ARGS}}' } + + run-paranoid: + desc: "Runs the server with paranoid security settings" + env: + VNLIB_SHARED_HEAP_GLOBAL_ZERO: 1 #force global heap zeroing + VNLIB_SHARED_HEAP_DIAGNOSTICS: 0 #disable heap diagnostics + + vars: + #enable zero allocation, even though global heap zero is set + # silent mode to reduce logging to the console + # force OS ciphers for TLS encryption + ARGS: "--config ../config/config.json + --zero-alloc + --silent + --use-os-ciphers + --input-off" + + cmds: + task: run + vars: { ARGS: '{{.ARGS}}' } + + run: + des: "Run the webserver with your specified command line arguments" + interactive: true + cmds: + #run the webserver as a cross-platform application + - dotnet VNLib.WebServer.dll {{.ARGS}} {{.CLI_ARGS}}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Bootstrap/ReleaseWebserver.cs b/apps/VNLib.WebServer/src/Bootstrap/ReleaseWebserver.cs new file mode 100644 index 0000000..543cbe5 --- /dev/null +++ b/apps/VNLib.WebServer/src/Bootstrap/ReleaseWebserver.cs @@ -0,0 +1,328 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ReleaseWebserver.cs +* +* ReleaseWebserver.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Data; +using System.Linq; +using System.Text; +using System.Text.Json; +using System.Collections.Generic; + +using VNLib.Utils.Logging; +using VNLib.Net.Http; +using VNLib.Plugins.Runtime; + +using VNLib.WebServer.Config; +using VNLib.WebServer.Config.Model; +using VNLib.WebServer.Plugins; +using VNLib.WebServer.Compression; +using VNLib.WebServer.Middlewares; +using VNLib.WebServer.RuntimeLoading; +using static VNLib.WebServer.Entry; + +namespace VNLib.WebServer.Bootstrap +{ + + /* + * This class represents a normally loaded "Relase" webserver to allow + * for module webserver use-cases. It relies on a system configuration + * file and command line arguments to configure the server. + */ + + internal class ReleaseWebserver(ServerLogger logger, IServerConfig config, ProcessArguments procArgs) + : WebserverBase(logger, config, procArgs) + { + + const string PLUGIN_DATA_TEMPLATE = +@" +---------------------------------- + | Plugin configuration: + | Enabled: {enabled} + | Directory: {dir} + | Hot Reload: {hr} + | Reload Delay: {delay}s +----------------------------------"; + + private readonly ProcessArguments args = procArgs; + + ///<inheritdoc/> + protected override PluginStackBuilder? ConfigurePlugins() + { + //do not load plugins if disabled + if (args.HasArgument("--no-plugins")) + { + logger.AppLog.Information("Plugin loading disabled via command-line flag"); + return null; + } + + JsonElement confEl = config.GetDocumentRoot(); + + if (!confEl.TryGetProperty(PLUGINS_CONFIG_PROP_NAME, out JsonElement plCfg)) + { + logger.AppLog.Debug("No plugin configuration found"); + return null; + } + + ServerPluginConfig? conf = plCfg.DeserializeElement<ServerPluginConfig>(); + Validate.EnsureNotNull(conf, "Your plugin configuration object is null or malformatted"); + + if (!conf.Enabled) + { + logger.AppLog.Information("Plugin loading disabled via configuration flag"); + return null; + } + + Validate.EnsureNotNull(conf.Path, "If plugins are enabled, you must specify a directory to load them from"); + + //Init new plugin stack builder + PluginStackBuilder pluginBuilder = PluginStackBuilder.Create() + .WithDebugLog(logger.AppLog) + .WithSearchDirectories([ conf.Path ]) + .WithLoaderFactory(PluginAsemblyLoading.Create); + + //Setup plugin config data + if (!string.IsNullOrWhiteSpace(conf.ConfigDir)) + { + pluginBuilder.WithJsonConfigDir(confEl, new(conf.ConfigDir)); + } + else + { + pluginBuilder.WithLocalJsonConfig(confEl); + } + + if (conf.HotReload) + { + Validate.EnsureRange(conf.ReloadDelaySec, 1, 120); + + pluginBuilder.EnableHotReload(TimeSpan.FromSeconds(conf.ReloadDelaySec)); + } + + logger.AppLog.Information( + PLUGIN_DATA_TEMPLATE, + true, + conf.Path, + conf.HotReload, + conf.ReloadDelaySec + ); + + if (conf.HotReload) + { + logger.AppLog.Warn("Plugin hot-reload is not recommended for production deployments!"); + } + + return pluginBuilder; + } + + ///<inheritdoc/> + protected override HttpConfig GetHttpConfig() + { + JsonElement rootEl = config.GetDocumentRoot(); + + try + { + HttpGlobalConfig? gConf = rootEl.GetProperty("http").DeserializeElement<HttpGlobalConfig>(); + Validate.EnsureNotNull(gConf, "Missing required HTTP configuration variables"); + + gConf.ValidateConfig(); + + //Attempt to load the compressor manager, if null, compression is disabled + IHttpCompressorManager? compressorManager = HttpCompressor.LoadOrDefaultCompressor(procArgs, gConf.Compression, config, logger.AppLog); + + IHttpMemoryPool memPool = MemoryPoolManager.GetHttpPool(procArgs.ZeroAllocations); + + HttpConfig conf = new(Encoding.ASCII) + { + ActiveConnectionRecvTimeout = gConf.RecvTimeoutMs, + CompressorManager = compressorManager, + ConnectionKeepAlive = TimeSpan.FromMilliseconds(gConf.KeepAliveMs), + CompressionLimit = gConf.Compression.CompressionMax, + CompressionMinimum = gConf.Compression.CompressionMin, + DebugPerformanceCounters = procArgs.HasArgument("--http-counters"), + DefaultHttpVersion = HttpHelpers.ParseHttpVersion(gConf.DefaultHttpVersion), + MaxFormDataUploadSize = gConf.MultipartMaxSize, + MaxUploadSize = gConf.MaxEntitySize, + MaxRequestHeaderCount = gConf.MaxRequestHeaderCount, + MaxOpenConnections = gConf.MaxConnections, + MaxUploadsPerRequest = gConf.MaxUploadsPerRequest, + SendTimeout = gConf.SendTimeoutMs, + ServerLog = logger.AppLog, + MemoryPool = memPool, + + RequestDebugLog = procArgs.LogHttp ? logger.AppLog : null, + + //Buffer config update + BufferConfig = new() + { + RequestHeaderBufferSize = gConf.HeaderBufSize, + ResponseHeaderBufferSize = gConf.ResponseHeaderBufSize, + FormDataBufferSize = gConf.MultipartMaxBufSize, + + //Align response buffer size with transport buffer to avoid excessive copy + ResponseBufferSize = TcpConfig.TcpTxBufferSize, + + /* + * Chunk buffers are written to the transport when they are fully accumulated. These buffers + * should be aligned with the transport sizes. It should also be large enough not to put too much + * back pressure on compressors. This buffer will be segmented into smaller buffers if it has to + * at the transport level, but we should avoid that if possible due to multiple allocations and + * copies. + * + * Aligning chunk buffer to the transport buffer size is the easiest solution to avoid excessive + * copyies + */ + ChunkedResponseAccumulatorSize = compressorManager != null ? TcpConfig.TcpTxBufferSize : 0 + }, + + }; + + Validate.Assert( + condition: conf.DefaultHttpVersion != HttpVersion.None, + message: "Your default HTTP version is invalid, specify an RFC formatted http version 'HTTP/x.x'" + ); + + return conf; + } + catch (KeyNotFoundException kne) + { + logger.AppLog.Error("Missing required HTTP configuration variables {var}", kne.Message); + throw new ServerConfigurationException("Missing required http variables. Cannot continue"); + } + } + + ///<inheritdoc/> + protected override VirtualHostConfig[] GetAllVirtualHosts() + { + ILogProvider log = logger.AppLog; + + LinkedList<VirtualHostConfig> configs = new(); + + try + { + int index = 0; + + //Enumerate all virtual host configurations + foreach (VirtualHostServerConfig vhConfig in GetVirtualHosts()) + { + + VirtualHostConfig conf = new JsonWebConfigBuilder(vhConfig, log).GetBaseConfig(); + + //Configure event hooks + conf.EventHooks = new VirtualHostHooks(conf); + + //Init middleware stack + conf.CustomMiddleware.Add(new MainServerMiddlware(log, conf, vhConfig.ForcePortCheck)); + + /* + * In benchmark mode, skip other middleware that might slow connections down + */ + if (vhConfig.Benchmark?.Enabled == true) + { + conf.CustomMiddleware.Add(new BenchmarkMiddleware(vhConfig.Benchmark)); + log.Information("BENCHMARK: Enabled for virtual host {vh}", conf.Hostnames); + } + else + { + /* + * We only enable cors if the configuration has a value for the allow cors property. + * The user may disable cors totally, deny cors requests, or enable cors with a whitelist + * + * Only add the middleware if the confg has a value for the allow cors property + */ + if (vhConfig.Cors?.Enabled == true) + { + conf.CustomMiddleware.Add(new CORSMiddleware(log, vhConfig.Cors)); + } + + //Add whitelist middleware if the configuration has a whitelist + if (conf.WhiteList != null) + { + conf.CustomMiddleware.Add(new IpWhitelistMiddleware(log, conf.WhiteList)); + } + + //Add blacklist middleware if the configuration has a blacklist + if (conf.BlackList != null) + { + conf.CustomMiddleware.Add(new IpBlacklistMiddleware(log, conf.BlackList)); + } + + //Add tracing middleware if enabled + if (vhConfig.RequestTrace) + { + conf.CustomMiddleware.Add(new ConnectionLogMiddleware(log)); + } + } + + if (!conf.RootDir.Exists) + { + conf.RootDir.Create(); + } + + configs.AddLast(conf); + + index++; + } + } + catch (KeyNotFoundException kne) + { + throw new ServerConfigurationException("Missing required configuration varaibles", kne); + } + catch (FormatException fe) + { + throw new ServerConfigurationException("Failed to parse IP address", fe); + } + + return configs.ToArray(); + } + + private VirtualHostServerConfig[] GetVirtualHosts() + { + JsonElement rootEl = config.GetDocumentRoot(); + ILogProvider log = logger.AppLog; + + if (!rootEl.TryGetProperty("virtual_hosts", out _)) + { + log.Warn("No virtual hosts array was defined. Continuing without hosts"); + return []; + } + + return rootEl.GetProperty("virtual_hosts") + .EnumerateArray() + .Select(GetVhConfig) + .ToArray(); + + + static VirtualHostServerConfig GetVhConfig(JsonElement rootEl) + { + VirtualHostServerConfig? conf = rootEl.DeserializeElement<VirtualHostServerConfig>(); + + Validate.EnsureNotNull(conf, "Empty virtual host configuration, check your virtual hosts array for an empty element"); + Validate.EnsureNotNull(conf.DirPath, "A virtual host was defined without a root directory property: 'dirPath'"); + Validate.EnsureNotNull(conf.Hostnames, "A virtual host was defined without a hostname property: 'hostnames'"); + Validate.EnsureNotNull(conf.Interfaces, "An interface configuration is required for every virtual host"); + + return conf; + } + } + } +} diff --git a/apps/VNLib.WebServer/src/Bootstrap/VariableLogFormatter.cs b/apps/VNLib.WebServer/src/Bootstrap/VariableLogFormatter.cs new file mode 100644 index 0000000..45d7bbb --- /dev/null +++ b/apps/VNLib.WebServer/src/Bootstrap/VariableLogFormatter.cs @@ -0,0 +1,57 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: VariableLogFormatter.cs +* +* VariableLogFormatter.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text; +using System.Collections.Generic; + +using VNLib.Utils.Logging; + +namespace VNLib.WebServer.Bootstrap +{ + internal sealed class VariableLogFormatter(ILogProvider logger, LogLevel level) + { + private readonly StringBuilder _logFormatSb = new(); + private readonly List<object?> _formatArgs = []; + + public void AppendLine(string line) => _logFormatSb.AppendLine(line); + + public void Append(string value) => _logFormatSb.Append(value); + + public void AppendFormat(string format, params object?[] formatargs) + { + _logFormatSb.Append(format); + _formatArgs.AddRange(formatargs); + } + + public void AppendLine() => _logFormatSb.AppendLine(); + + public void Flush() + { + logger.Write(level, _logFormatSb.ToString(), [.._formatArgs]); + + _logFormatSb.Clear(); + _formatArgs.Clear(); + } + } +} diff --git a/apps/VNLib.WebServer/src/Bootstrap/WebserverBase.cs b/apps/VNLib.WebServer/src/Bootstrap/WebserverBase.cs new file mode 100644 index 0000000..f3832c6 --- /dev/null +++ b/apps/VNLib.WebServer/src/Bootstrap/WebserverBase.cs @@ -0,0 +1,203 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: WebserverBase.cs +* +* WebserverBase.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Linq; +using System.Text.Json; +using System.Diagnostics; + +using VNLib.Net.Http; +using VNLib.Utils; +using VNLib.Utils.Extensions; +using VNLib.Plugins.Runtime; +using VNLib.Plugins.Essentials.ServiceStack; +using VNLib.Plugins.Essentials.ServiceStack.Construction; + +using VNLib.WebServer.Config; +using VNLib.WebServer.Transport; +using VNLib.WebServer.RuntimeLoading; + +namespace VNLib.WebServer.Bootstrap +{ + + internal abstract class WebserverBase(ServerLogger logger, IServerConfig config, ProcessArguments procArgs) + : VnDisposeable + { + + protected readonly ProcessArguments procArgs = procArgs; + protected readonly IServerConfig config = config; + protected readonly ServerLogger logger = logger; + protected readonly TcpServerLoader TcpConfig = new(config, procArgs, logger.SysLog); + + private HttpServiceStack? _serviceStack; + + /// <summary> + /// Gets the internal <see cref="HttpServiceStack"/> this + /// controller is managing + /// </summary> + public HttpServiceStack ServiceStack + { + get + { + if (_serviceStack is null) + { + throw new InvalidOperationException("Service stack has not been configured yet"); + } + + return _serviceStack; + } + } + + /// <summary> + /// Configures the http server for the application so + /// its ready to start + /// </summary> + public virtual void Configure() + { + _serviceStack = ConfiugreServiceStack(); + } + + protected virtual HttpServiceStack ConfiugreServiceStack() + { + bool loadPluginsConcurrently = !procArgs.HasArgument("--sequential-load"); + + JsonElement conf = config.GetDocumentRoot(); + + HttpConfig http = GetHttpConfig(); + + VirtualHostConfig[] virtualHosts = GetAllVirtualHosts(); + + PluginStackBuilder? plugins = ConfigurePlugins(); + + HttpServiceStackBuilder builder = new HttpServiceStackBuilder() + .LoadPluginsConcurrently(loadPluginsConcurrently) + .WithBuiltInHttp(TcpConfig.ReduceBindingsForGroups, http) + .WithDomain(domain => + { + domain.WithServiceGroups(vh => + { + /* + * Must pass the virtual host configuration as the state object + * so transport providers can be loaded from a given virtual host + */ + virtualHosts.ForEach(vhConfig => vh.WithVirtualHost(vhConfig, vhConfig)); + }); + }); + + if (plugins != null) + { + builder.WithPluginStack(plugins.ConfigureStack); + } + + PrintLogicalRouting(virtualHosts); + + return builder.Build(); + } + + protected abstract VirtualHostConfig[] GetAllVirtualHosts(); + + protected abstract HttpConfig GetHttpConfig(); + + protected abstract PluginStackBuilder? ConfigurePlugins(); + + /// <summary> + /// Starts the server and returns immediately + /// after server start listening + /// </summary> + public void Start() + { + /* Since this api is uses internally, knowing the order of operations is a bug, not a rumtime accident */ + Debug.Assert(Disposed == false, "Server was disposed"); + Debug.Assert(_serviceStack != null, "Server was not configured"); + + //Attempt to load plugins before starting server + _serviceStack.LoadPlugins(logger.AppLog); + + _serviceStack.StartServers(); + } + + /// <summary> + /// Stops the server and waits for all connections to close and + /// servers to fully shut down + /// </summary> + public void Stop() + { + Debug.Assert(Disposed == false, "Server was disposed"); + Debug.Assert(_serviceStack != null, "Server was not configured"); + + //Stop the server and wait synchronously + _serviceStack.StopAndWaitAsync() + .GetAwaiter() + .GetResult(); + } + + private void PrintLogicalRouting(VirtualHostConfig[] hosts) + { + const string header =@" +=================================================== + --- HTTP Service Domain --- + + {enabledRoutes} routes enabled +"; + + VariableLogFormatter sb = new(logger.AppLog, Utils.Logging.LogLevel.Information); + sb.AppendFormat(header, hosts.Length); + + foreach (VirtualHostConfig host in hosts) + { + sb.AppendLine(); + + sb.AppendFormat("Virtual Host: {hostnames}\n", (object)host.Hostnames); + sb.AppendFormat(" Root directory {rdir}\n", host.RootDir); + sb.AppendLine(); + + //Print interfaces + + string[] interfaces = host.Transports + .Select(i =>$" - {i.Address}:{i.Port} TLS: {i.Ssl}, Client cert: {i.ClientCertRequired}, OS Ciphers: {i.UseOsCiphers}") + .ToArray(); + + sb.AppendLine(" Interfaces:"); + sb.AppendFormat("{interfaces}", string.Join("\n", interfaces)); + sb.AppendLine(); + + sb.AppendLine(" Options:"); + sb.AppendFormat(" - Whitelist: {wl}\n", host.WhiteList); + sb.AppendFormat(" - Blacklist: {bl}\n", host.BlackList); + sb.AppendFormat(" - Path filter: {filter}\n", host.PathFilter); + sb.AppendFormat(" - Cache default time: {cache}\n", host.CacheDefault); + sb.AppendFormat(" - Cached error files: {files}\n", host.FailureFiles.Select(static p => (int)p.Key)); + sb.AppendFormat(" - Downstream servers: {dsServers}\n", host.DownStreamServers); + sb.AppendFormat(" - Middlewares loaded {mw}\n", host.CustomMiddleware.Count); + sb.AppendLine(); + + sb.Flush(); + } + } + + + ///<inheritdoc/> + protected override void Free() => _serviceStack?.Dispose(); + } +} diff --git a/apps/VNLib.WebServer/src/CommandListener.cs b/apps/VNLib.WebServer/src/CommandListener.cs new file mode 100644 index 0000000..7082d40 --- /dev/null +++ b/apps/VNLib.WebServer/src/CommandListener.cs @@ -0,0 +1,251 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: CommandListener.cs +* +* CommandListener.cs is part of VNLib.WebServer which is part of +* the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Threading; + +using VNLib.Utils.Memory; +using VNLib.Utils.Logging; +using VNLib.Utils.Extensions; +using VNLib.Utils.Memory.Diagnostics; +using VNLib.Net.Http; +using VNLib.Plugins.Essentials.ServiceStack; +using VNLib.Plugins.Essentials.ServiceStack.Plugins; + +using VNLib.WebServer.Bootstrap; + +namespace VNLib.WebServer +{ + + internal sealed class CommandListener(ManualResetEvent shutdownEvent, WebserverBase server, ILogProvider log) + { + const string MANAGED_HEAP_STATS = @" + Managed Heap Stats +-------------------------------------- + Collections: + Gen0: {g0} Gen1: {g1} Gen2: {g2} + + Heap: + High Watermark: {hw} KB + Last GC Heap Size: {hz} KB + Current Load: {ld} KB + Fragmented: {fb} KB + + Heap Info: + Last GC concurrent? {con} + Last GC compacted? {comp} + Pause time: {pt} % + Pending finalizers: {pf} + Pinned objects: {po} +"; + + const string HEAPSTATS = @" + Unmanaged Heap Stats +--------------------------- + userHeap? {rp} + Allocated bytes: {ab} + Allocated handles: {h} + Max block size: {mb} + Min block size: {mmb} + Max heap size: {hs} +"; + const string HELP = @" + VNLib.WebServer console help menu + + p <plugin-name> <command> - Sends a command to a plugin + cmd <plugin-name> - Enters a command loop for the specified plugin + reload - Reloads all plugins + memstats - Prints memory stats + collect - Flushes server caches, collects, and compacts memory + stop - Stops the server + help - Prints this help menu +"; + + + private readonly HttpServiceStack _serviceStack = server.ServiceStack; + private readonly IHttpPluginManager _plugins = server.ServiceStack.PluginManager; + + + /// <summary> + /// Listens for commands and processes them in a continuous loop + /// </summary> + /// <param name="shutdownEvent">A <see cref="ManualResetEvent"/> that is set when the Stop command is received</param> + /// <param name="server">The webserver for the current process</param> + public void ListenForCommands(TextReader input, TextWriter output, string name) + { + log.Information("Listening for commands on {con}", name); + + while (shutdownEvent.WaitOne(0) == false) + { + string[]? s = input.ReadLine()?.Split(' '); + if (s == null) + { + continue; + } + switch (s[0].ToLower(null)) + { + case "help": + output.WriteLine(HELP); + break; + //handle plugin + case "p": + { + if (s.Length < 3) + { + output.WriteLine("Plugin name and command are required"); + break; + } + + string message = string.Join(' ', s[2..]); + + bool sent = _plugins.SendCommandToPlugin(s[1], message, StringComparison.OrdinalIgnoreCase); + + if (!sent) + { + output.WriteLine("Plugin not found"); + } + } + break; + + case "cmd": + { + if (s.Length < 2) + { + output.WriteLine("Plugin name is required"); + break; + } + + //Enter plugin command loop + EnterPluginLoop(input, output, s[1], _plugins); + } + break; + case "reload": + { + try + { + //Reload all plugins + _plugins.ForceReloadAllPlugins(); + } + catch (Exception ex) + { + log.Error(ex); + } + } + break; + case "memstats": + { + + + //Collect gc info for managed heap stats + int gen0 = GC.CollectionCount(0); + int gen1 = GC.CollectionCount(1); + int gen2 = GC.CollectionCount(2); + GCMemoryInfo mi = GC.GetGCMemoryInfo(); + + log.Debug(MANAGED_HEAP_STATS, + gen0, + gen1, + gen2, + mi.HighMemoryLoadThresholdBytes / 1024, + mi.HeapSizeBytes / 1024, + mi.MemoryLoadBytes / 1024, + mi.FragmentedBytes / 1024, + mi.Concurrent, + mi.Compacted, + mi.PauseTimePercentage, + mi.FinalizationPendingCount, + mi.PinnedObjectsCount + ); + + //Get heap stats + HeapStatistics hs = MemoryUtil.GetSharedHeapStats(); + + //Print unmanaged heap stats + log.Debug(HEAPSTATS, + MemoryUtil.IsUserDefinedHeap, + hs.AllocatedBytes, + hs.AllocatedBlocks, + hs.MaxBlockSize, + hs.MinBlockSize, + hs.MaxHeapSize + ); + } + break; + case "collect": + CollectCache(_serviceStack); + GC.Collect(2, GCCollectionMode.Forced, false, true); + GC.WaitForFullGCComplete(); + break; + case "stop": + shutdownEvent.Set(); + return; + } + } + } + + /* + * Function scopes commands as if the user is writing directly to + * the plugin. All commands are passed to the plugin manager for + * processing. + */ + private static void EnterPluginLoop( + TextReader input, + TextWriter output, + string pluignName, + IHttpPluginManager man + ) + { + output.WriteLine("Entering plugin {0}. Type 'exit' to leave", pluignName); + + while (true) + { + output.Write("{0}>", pluignName); + + string? cmdText = input.ReadLine(); + + if (string.IsNullOrWhiteSpace(cmdText)) + { + output.WriteLine("Please enter a command or type 'exit' to leave"); + continue; + } + + if (string.Equals(cmdText, "exit", StringComparison.OrdinalIgnoreCase)) + { + break; + } + + //Exec command + if (!man.SendCommandToPlugin(pluignName, cmdText, StringComparison.OrdinalIgnoreCase)) + { + output.WriteLine("Plugin does not exist exiting loop"); + break; + } + } + } + + private static void CollectCache(HttpServiceStack controller) + => controller.Servers.ForEach(static server => (server as HttpServer)!.CacheClear()); + } +} diff --git a/apps/VNLib.WebServer/src/Compression/FallbackCompressionManager.cs b/apps/VNLib.WebServer/src/Compression/FallbackCompressionManager.cs new file mode 100644 index 0000000..d2eb719 --- /dev/null +++ b/apps/VNLib.WebServer/src/Compression/FallbackCompressionManager.cs @@ -0,0 +1,144 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: FallbackCompressionManager.cs +* +* FallbackCompressionManager.cs is part of VNLib.WebServer which is part +* of the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Buffers; +using System.Diagnostics; +using System.IO.Compression; + +using VNLib.Net.Http; + +namespace VNLib.WebServer.Compression +{ + + /* + * The fallback compression manager is used when the user did not configure a + * compression manager library. Since .NET only exposes a brotli encoder, that + * is not a stream api, (gzip and deflate are stream api's) Im only supporting + * brotli for now. This is better than nothing lol + */ + + + internal sealed class FallbackCompressionManager : IHttpCompressorManager + { + /// <inheritdoc/> + public object AllocCompressor() => new BrCompressorState(); + + /// <inheritdoc/> + public CompressionMethod GetSupportedMethods() => CompressionMethod.Brotli; + + /// <inheritdoc/> + public int InitCompressor(object compressorState, CompressionMethod compMethod) + { + BrCompressorState compressor = (BrCompressorState)compressorState; + ref BrotliEncoder encoder = ref compressor.GetEncoder(); + + //Init new brotli encoder struct + encoder = new(9, 24); + return 0; + } + + /// <inheritdoc/> + public void DeinitCompressor(object compressorState) + { + BrCompressorState compressor = (BrCompressorState)compressorState; + ref BrotliEncoder encoder = ref compressor.GetEncoder(); + + //Clean up the encoder + encoder.Dispose(); + encoder = default; + } + + /// <inheritdoc/> + public CompressionResult CompressBlock(object compressorState, ReadOnlyMemory<byte> input, Memory<byte> output) + { + //Output buffer should never be empty, server guards this + Debug.Assert(!output.IsEmpty, "Exepcted a non-zero length output buffer"); + + BrCompressorState compressor = (BrCompressorState)compressorState; + ref BrotliEncoder encoder = ref compressor.GetEncoder(); + + //Compress the supplied block + OperationStatus status = encoder.Compress(input.Span, output.Span, out int bytesConsumed, out int bytesWritten, false); + + /* + * Should always return done, because the output buffer is always + * large enough and that data/state cannot be invalid + */ + Debug.Assert(status == OperationStatus.Done); + + return new() + { + BytesRead = bytesConsumed, + BytesWritten = bytesWritten, + }; + } + + /// <inheritdoc/> + public int Flush(object compressorState, Memory<byte> output) + { + OperationStatus status; + + //Output buffer should never be empty, server guards this + Debug.Assert(!output.IsEmpty, "Exepcted a non-zero length output buffer"); + + BrCompressorState compressor = (BrCompressorState)compressorState; + ref BrotliEncoder encoder = ref compressor.GetEncoder(); + + /* + * A call to compress with the isFinalBlock flag set to true will + * cause a BROTLI_OPERATION_FINISH operation to be performed. This is + * actually the proper way to complete a brotli compression stream. + * + * See vnlib_compress project for more details. + */ + status = encoder.Compress( + source: default, + destination: output.Span, + bytesConsumed: out _, + bytesWritten: out int bytesWritten, + isFinalBlock: true + ); + + /* + * Function can return Done or DestinationTooSmall if there is still more data + * stored in the compressor to be written. If InvaliData is returned, then there + * is a problem with the encoder state or the output buffer, this condition should + * never happen. + */ + Debug.Assert(status != OperationStatus.InvalidData, $"Failed with status {status}, written {bytesWritten}, buffer size {output.Length}"); + + //Return the number of bytes actually accumulated + return bytesWritten; + } + + + private sealed class BrCompressorState + { + private BrotliEncoder _encoder; + + public ref BrotliEncoder GetEncoder() => ref _encoder; + } + } +} diff --git a/apps/VNLib.WebServer/src/Compression/HttpCompressor.cs b/apps/VNLib.WebServer/src/Compression/HttpCompressor.cs new file mode 100644 index 0000000..beda67b --- /dev/null +++ b/apps/VNLib.WebServer/src/Compression/HttpCompressor.cs @@ -0,0 +1,123 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: HttpCompressor.cs +* +* HttpCompressor.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Text.Json; +using System.Reflection; +using System.Runtime.Loader; + +using VNLib.Utils.IO; +using VNLib.Utils.Logging; +using VNLib.Utils.Resources; +using VNLib.Net.Http; + +using VNLib.WebServer.Config; +using VNLib.WebServer.RuntimeLoading; +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer.Compression +{ + + internal static class HttpCompressor + { + /* + * A function delegate that is invoked on the user-defined http compressor library + * when loaded + */ + private delegate void OnHttpLibLoad(ILogProvider log, JsonElement? configData); + + /// <summary> + /// Attempts to load a user-defined http compressor library from the specified path in the config, + /// otherwise falls back to the default http compressor, unless the command line disabled compression. + /// </summary> + /// <param name="args">Process wide- argument list</param> + /// <param name="config">The top-level config element</param> + /// <param name="logger">The application logger to write logging events to</param> + /// <returns>The <see cref="IHttpCompressorManager"/> that the user configured, or null if disabled</returns> + public static IHttpCompressorManager? LoadOrDefaultCompressor(ProcessArguments args, HttpCompressorConfig compConfig, IServerConfig config, ILogProvider logger) + { + const string EXTERN_LIB_LOAD_METHOD_NAME = "OnLoad"; + + if (args.HasArgument("--compression-off")) + { + logger.Information("Http compression disabled by cli args"); + return null; + } + + if(!compConfig.Enabled) + { + logger.Information("Http compression disabled by config"); + return null; + } + + if (string.IsNullOrWhiteSpace(compConfig.AssemblyPath)) + { + logger.Information("Falling back to default http compressor"); + return new FallbackCompressionManager(); + } + + //Make sure the file exists + if (!FileOperations.FileExists(compConfig.AssemblyPath)) + { + logger.Warn("The specified http compressor assembly file does not exist, falling back to default http compressor"); + return new FallbackCompressionManager(); + } + + //Try to load the assembly into our process alc, we dont need to worry about unloading + ManagedLibrary lib = ManagedLibrary.LoadManagedAssembly(compConfig.AssemblyPath, AssemblyLoadContext.Default); + + logger.Debug("Loading user defined compressor assembly: {asm}", Path.GetFileName(lib.AssemblyPath)); + + try + { + //Load the compressor manager type from the assembly + IHttpCompressorManager instance = lib.LoadTypeFromAssembly<IHttpCompressorManager>(); + + /* + * We can provide some optional library initialization functions if the library + * supports it. First we can allow the library to write logs to our log provider + * and second we can provide the library with the raw configuration data as a byte array + */ + + //Invoke the on load method with the logger and config data + OnHttpLibLoad? onlibLoadConfig = ManagedLibrary.TryGetMethod<OnHttpLibLoad>(instance, EXTERN_LIB_LOAD_METHOD_NAME); + onlibLoadConfig?.Invoke(logger, config.GetDocumentRoot()); + + //Invoke parameterless on load method + Action? onLibLoad = ManagedLibrary.TryGetMethod<Action>(instance, EXTERN_LIB_LOAD_METHOD_NAME); + onLibLoad?.Invoke(); + + logger.Information("Custom compressor library loaded"); + + return instance; + } + //Catch TIE and throw the inner exception for cleaner debug + catch (TargetInvocationException te) when (te.InnerException != null) + { + throw te.InnerException; + } + } + } +} diff --git a/apps/VNLib.WebServer/src/Config/IServerConfig.cs b/apps/VNLib.WebServer/src/Config/IServerConfig.cs new file mode 100644 index 0000000..b0b06ba --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/IServerConfig.cs @@ -0,0 +1,34 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: IServerConfig.cs +* +* IServerConfig.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json; + + +namespace VNLib.WebServer.Config +{ + internal interface IServerConfig + { + JsonElement GetDocumentRoot(); + } +} diff --git a/apps/VNLib.WebServer/src/Config/JsonConfigOptions.cs b/apps/VNLib.WebServer/src/Config/JsonConfigOptions.cs new file mode 100644 index 0000000..a48b4e9 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/JsonConfigOptions.cs @@ -0,0 +1,42 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: JsonConfigOptions.cs +* +* JsonConfigOptions.cs is part of VNLib.WebServer which is part of +* the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json; + +namespace VNLib.WebServer.Config +{ + internal static class JsonConfigOptions + { + private static readonly JsonSerializerOptions _ops = new() + { + AllowTrailingCommas = true, + ReadCommentHandling = JsonCommentHandling.Skip, + }; + + public static T? DeserializeElement<T>(this JsonElement el) + { + return el.Deserialize<T>(_ops); + } + } +} diff --git a/apps/VNLib.WebServer/src/Config/JsonServerConfig.cs b/apps/VNLib.WebServer/src/Config/JsonServerConfig.cs new file mode 100644 index 0000000..ada9902 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/JsonServerConfig.cs @@ -0,0 +1,156 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: JsonServerConfig.cs +* +* JsonServerConfig.cs is part of VNLib.WebServer which is part +* of the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Text.Json; + +using YamlDotNet.Core.Events; +using YamlDotNet.Serialization; + +using VNLib.Utils.IO; + +namespace VNLib.WebServer.Config +{ + internal sealed class JsonServerConfig(JsonDocument doc) : IServerConfig + { + public JsonElement GetDocumentRoot() => doc.RootElement; + + public static JsonServerConfig? FromFile(string filename) + { + string nameOnly = Path.GetFileName(filename); + Console.WriteLine("Loading configuration file from {0}", nameOnly); + + if (filename.EndsWith(".json")) + { + return FromJson(filename); + } + else if (filename.EndsWith(".yaml") || filename.EndsWith(".yml")) + { + return FromYaml(filename); + } + else + { + return null; + } + } + + /// <summary> + /// Reads a server configuration from the specified JSON document + /// </summary> + /// <param name="configPath">The file path of the json cofiguration file</param> + /// <returns>A new <see cref="JsonServerConfig"/> wrapping the server config</returns> + public static JsonServerConfig? FromJson(string fileName) + { + if (!FileOperations.FileExists(fileName)) + { + return null; + } + + //Open the config file + using FileStream fs = File.OpenRead(fileName); + + //Allow comments + JsonDocumentOptions jdo = new() + { + CommentHandling = JsonCommentHandling.Skip, + AllowTrailingCommas = true, + }; + + return new JsonServerConfig(JsonDocument.Parse(fs, jdo)); + } + + public static JsonServerConfig? FromYaml(string fileName) + { + if (!FileOperations.FileExists(fileName)) + { + return null; + } + + /* + * The following code reads the configuration as a yaml + * object and then serializes it over to json. + */ + + using StreamReader reader = OpenFileRead(fileName); + + object? yamlObject = new DeserializerBuilder() + .WithNodeTypeResolver(new NumberTypeResolver()) + .Build() + .Deserialize(reader); + + ISerializer serializer = new SerializerBuilder() + .JsonCompatible() + .Build(); + + using VnMemoryStream ms = new(); + using (StreamWriter sw = new(ms, leaveOpen: true)) + { + serializer.Serialize(sw, yamlObject); + } + + ms.Seek(0, SeekOrigin.Begin); + + return new JsonServerConfig(JsonDocument.Parse(ms)); + } + + private static StreamReader OpenFileRead(string fileName) + { + return new StreamReader( + stream: File.OpenRead(fileName), + encoding: System.Text.Encoding.UTF8, + detectEncodingFromByteOrderMarks: false, + leaveOpen: false + ); + } + + public class NumberTypeResolver : INodeTypeResolver + { + public bool Resolve(NodeEvent? nodeEvent, ref Type currentType) + { + if (nodeEvent is Scalar scalar) + { + if(long.TryParse(scalar.Value, out _)) + { + currentType = typeof(int); + return true; + } + + if (double.TryParse(scalar.Value, out _)) + { + currentType = typeof(double); + return true; + } + + if (bool.TryParse(scalar.Value, out _)) + { + currentType = typeof(bool); + return true; + } + } + return false; + } + } + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/BenchmarkConfig.cs b/apps/VNLib.WebServer/src/Config/Model/BenchmarkConfig.cs new file mode 100644 index 0000000..1b0f157 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/BenchmarkConfig.cs @@ -0,0 +1,43 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: BenchmarkConfig.cs +* +* BenchmarkConfig.cs is part of VNLib.WebServer which is part of the +* larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + + internal sealed class BenchmarkConfig + { + + [JsonPropertyName("enabled")] + public bool Enabled { get; set; } + + [JsonPropertyName("size")] + public int Size { get; set; } + + [JsonPropertyName("random")] + public bool Random { get; set; } + + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/CorsSecurityConfig.cs b/apps/VNLib.WebServer/src/Config/Model/CorsSecurityConfig.cs new file mode 100644 index 0000000..2f95697 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/CorsSecurityConfig.cs @@ -0,0 +1,41 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: CorsSecurityConfig.cs +* +* CorsSecurityConfig.cs is part of VNLib.WebServer which is part of the +* larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal class CorsSecurityConfig + { + [JsonPropertyName("enabled")] + public bool Enabled { get; set; } = false; + + [JsonPropertyName("deny_cors_connections")] + public bool DenyCorsCons { get; set; } = false; + + [JsonPropertyName("allowed_authority")] + public string[] AllowedCorsAuthority { get; set; } = Array.Empty<string>(); + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Config/Model/ErrorFileConfig.cs b/apps/VNLib.WebServer/src/Config/Model/ErrorFileConfig.cs new file mode 100644 index 0000000..c4355d6 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/ErrorFileConfig.cs @@ -0,0 +1,37 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ErrorFileConfig.cs +* +* ErrorFileConfig.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal sealed class ErrorFileConfig + { + [JsonPropertyName("code")] + public int Code { get; set; } + + [JsonPropertyName("path")] + public string? Path { get; set; } + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/HttpCompressorConfig.cs b/apps/VNLib.WebServer/src/Config/Model/HttpCompressorConfig.cs new file mode 100644 index 0000000..eb8e68c --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/HttpCompressorConfig.cs @@ -0,0 +1,47 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: HttpCompressorConfig.cs +* +* HttpCompressorConfig.cs is part of VNLib.WebServer which is part of +* the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal sealed class HttpCompressorConfig + { + [JsonPropertyName("assembly")] + public string? AssemblyPath { get; set; } + + /// <summary> + /// If this compressor is enabled. The default is true, to use built-in + /// compressors. + /// </summary> + [JsonPropertyName("enabled")] + public bool Enabled { get; set; } = true; + + [JsonPropertyName("max_size")] + public long CompressionMax { get; set; } = 104857600; //100MB + + [JsonPropertyName("min_size")] + public int CompressionMin { get; set; } = 256; + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/HttpGlobalConfig.cs b/apps/VNLib.WebServer/src/Config/Model/HttpGlobalConfig.cs new file mode 100644 index 0000000..22bfe75 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/HttpGlobalConfig.cs @@ -0,0 +1,137 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: HttpGlobalConfig.cs +* +* HttpGlobalConfig.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal class HttpGlobalConfig + { + + [JsonPropertyName("default_version")] + public string DefaultHttpVersion { get; set; } = "HTTP/1.1"; + + /// <summary> + /// The maximum size of a request entity that can be sent to the server. + /// </summary> + [JsonPropertyName("max_entity_size")] + public long MaxEntitySize { get; set; } = long.MaxValue; + + /// <summary> + /// The maximum size of a multipart form data upload. + /// </summary> + [JsonPropertyName("multipart_max_size")] + public int MultipartMaxSize { get; set; } = 1048576; //1MB + + /// <summary> + /// The time in milliseconds for an HTTP/1.1 connection to remain open + /// before the server closes it. + /// </summary> + [JsonPropertyName("keepalive_ms")] + public int KeepAliveMs { get; set; } = 60000; //60 seconds + + /// <summary> + /// The time in milliseconds to wait for data on an active connection. + /// IE: A connection that has been established and has signaled that + /// it is ready to transfer data. + /// </summary> + [JsonPropertyName("recv_timeout_ms")] + public int RecvTimeoutMs { get; set; } = 5000; //5 seconds + + /// <summary> + /// The time in milliseconds to wait for data to be sent on a connection. + /// </summary> + [JsonPropertyName("send_timeout_ms")] + public int SendTimeoutMs { get; set; } = 60000; //60 seconds + + /// <summary> + /// The maximum number of headers that can be sent in a request. + /// </summary> + [JsonPropertyName("max_request_header_count")] + public int MaxRequestHeaderCount { get; set; } = 32; + + /// <summary> + /// The maximum number of open connections that can be made to the server, before + /// the server starts rejecting new connections. + /// </summary> + [JsonPropertyName("max_connections")] + public int MaxConnections { get; set; } = int.MaxValue; + + /// <summary> + /// The maximum number of uploads that can be made in a single request. If + /// this value is exceeded, the request will be rejected. + /// </summary> + [JsonPropertyName("max_uploads_per_request")] + public ushort MaxUploadsPerRequest { get; set; } = 10; + + /// <summary> + /// The size of the buffer used to store request headers. + /// </summary> + [JsonPropertyName("header_buf_size")] + public int HeaderBufSize { get; set; } + + /// <summary> + /// The size of the buffer used to store response headers. + /// </summary> + [JsonPropertyName("response_header_buf_size")] + public int ResponseHeaderBufSize { get; set; } + + /// <summary> + /// The size of the buffer used to store form data. + /// </summary> + [JsonPropertyName("multipart_max_buf_size")] + public int MultipartMaxBufSize { get; set; } + + /// <summary> + /// The configuration for the HTTP compression settings. + /// </summary> + [JsonPropertyName("compression")] + public HttpCompressorConfig? Compression { get; set; } = new(); + + public void ValidateConfig() + { + Validate.EnsureNotNull(DefaultHttpVersion, "Default HTTP version is required"); + + Validate.EnsureRange(MaxEntitySize, 0, long.MaxValue); + Validate.EnsureRange(MultipartMaxSize, -1, int.MaxValue); + Validate.EnsureRange(KeepAliveMs, -1, int.MaxValue); + + //Timeouts may be disabled by setting 0 or -1. Both are allowed for readability + Validate.EnsureRange(RecvTimeoutMs, -2, int.MaxValue); + Validate.EnsureRange(SendTimeoutMs, -2, int.MaxValue); + + Validate.EnsureRange(MaxRequestHeaderCount, 0, 1024); + Validate.EnsureRange(MaxConnections, 0, int.MaxValue); + Validate.EnsureRange(MaxUploadsPerRequest, 0, 1024); + Validate.EnsureRange(HeaderBufSize, 0, int.MaxValue); + Validate.EnsureRange(ResponseHeaderBufSize, 0, int.MaxValue); + Validate.EnsureRange(MultipartMaxBufSize, 0, int.MaxValue); + + //Validate compression config + Validate.EnsureNotNull(Compression, "Compression configuration should not be set to null. Comment to enable defaults"); + Validate.EnsureRange(Compression.CompressionMax, -1, long.MaxValue); + Validate.EnsureRange(Compression.CompressionMin, -1, int.MaxValue); + } + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/LogConfig.cs b/apps/VNLib.WebServer/src/Config/Model/LogConfig.cs new file mode 100644 index 0000000..fafd630 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/LogConfig.cs @@ -0,0 +1,52 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: LogConfig.cs +* +* LogConfig.cs is part of VNLib.WebServer which is part of the +* larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal sealed class LogConfig + { + [JsonPropertyName("enabled")] + public bool Enabled { get; set; } = false; + + [JsonPropertyName("path")] + public string? Path { get; set; } + + [JsonPropertyName("template")] + public string? Template { get; set; } + + [JsonPropertyName("flush_sec")] + public int FlushIntervalSeconds { get; set; } = 10; + + [JsonPropertyName("retained_files")] + public int RetainedFiles { get; set; } = 31; + + [JsonPropertyName("file_size_limit")] + public int FileSizeLimit { get; set; } = 500 * 1000 * 1024; + + [JsonPropertyName("interval")] + public string Interval { get; set; } = "infinite"; + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/ServerPluginConfig.cs b/apps/VNLib.WebServer/src/Config/Model/ServerPluginConfig.cs new file mode 100644 index 0000000..42b91b1 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/ServerPluginConfig.cs @@ -0,0 +1,46 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ServerPluginConfig.cs +* +* ServerPluginConfig.cs is part of VNLib.WebServer which is part of +* the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal class ServerPluginConfig + { + [JsonPropertyName("enabled")] + public bool Enabled { get; set; } = true; //default to true if config is defined, then it can be assumed we want to load plugins unless explicitly disabled + + [JsonPropertyName("path")] + public string? Path { get; set; } + + [JsonPropertyName("config_dir")] + public string? ConfigDir { get; set; } + + [JsonPropertyName("hot_reload")] + public bool HotReload { get; set; } + + [JsonPropertyName("reload_delay_sec")] + public int ReloadDelaySec { get; set; } = 2; + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/TcpConfigJson.cs b/apps/VNLib.WebServer/src/Config/Model/TcpConfigJson.cs new file mode 100644 index 0000000..5bd2b94 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/TcpConfigJson.cs @@ -0,0 +1,73 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: TcpServerLoader.cs +* +* TcpServerLoader.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal sealed class TcpConfigJson + { + [JsonPropertyName("keepalive_sec")] + public int TcpKeepAliveTime { get; set; } = 4; + + [JsonPropertyName("keepalive_interval_sec")] + public int KeepaliveInterval { get; set; } = 4; + + [JsonPropertyName("max_recv_buffer")] + public int MaxRecvBufferData { get; set; } = 10 * 64 * 1024; + + [JsonPropertyName("backlog")] + public int BackLog { get; set; } = 1000; + + [JsonPropertyName("max_connections")] + public long MaxConnections { get; set; } = long.MaxValue; + + [JsonPropertyName("no_delay")] + public bool NoDelay { get; set; } = false; + + /* + * Buffer sizes are a pain, this is a good default size for medium bandwith connections (100mbps) + * using the BDP calculations + * + * BDP = Bandwidth * RTT + */ + + [JsonPropertyName("tx_buffer")] + public int TcpSendBufferSize { get; set; } = 625 * 1024; + + [JsonPropertyName("rx_buffer")] + public int TcpRecvBufferSize { get; set; } = 625 * 1024; + + + public void ValidateConfig() + { + Validate.EnsureRange(TcpKeepAliveTime, 0, 60); + Validate.EnsureRange(KeepaliveInterval, 0, 60); + Validate.EnsureRange(BackLog, 0, 10000); + Validate.EnsureRange(MaxConnections, 0, long.MaxValue); + Validate.EnsureRange(MaxRecvBufferData, 0, 10 * 1024 * 1024); //10MB + Validate.EnsureRange(TcpSendBufferSize, 0, 10 * 1024 * 1024); //10MB + } + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/TransportInterface.cs b/apps/VNLib.WebServer/src/Config/Model/TransportInterface.cs new file mode 100644 index 0000000..e0e2551 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/TransportInterface.cs @@ -0,0 +1,137 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: TransportInterface.cs +* +* TransportInterface.cs is part of VNLib.WebServer which is part of +* the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Net; +using System.Security.Cryptography.X509Certificates; +using System.Text.Json.Serialization; + +using VNLib.Utils.Memory; +using VNLib.Utils.Resources; + +namespace VNLib.WebServer.Config.Model +{ + /// <summary> + /// Represents a transport interface configuration element for a virtual host + /// </summary> + internal class TransportInterface + { + [JsonPropertyName("port")] + public int Port { get; set; } + + [JsonPropertyName("address")] + public string? Address { get; set; } + + [JsonPropertyName("certificate")] + public string? Cert { get; set; } + + [JsonPropertyName("private_key")] + public string? PrivKey { get; set; } + + [JsonPropertyName("ssl")] + public bool Ssl { get; set; } + + [JsonPropertyName("client_cert_required")] + public bool ClientCertRequired { get; set; } + + [JsonPropertyName("password")] + public string? PrivKeyPassword { get; set; } + + [JsonPropertyName("use_os_ciphers")] + public bool UseOsCiphers { get; set; } + + public IPEndPoint GetEndpoint() + { + IPAddress addr = string.IsNullOrEmpty(Address) ? IPAddress.Any : IPAddress.Parse(Address); + return new IPEndPoint(addr, Port); + } + + public X509Certificate? LoadCertificate() + { + if (!Ssl) + { + return null; + } + + Validate.EnsureNotNull(Cert, "TLS Certificate is required when ssl is enabled"); + Validate.FileExists(Cert); + + X509Certificate? cert = null; + + /* + * Default to use a PEM encoded certificate and private key file. Unless the file + * is a pfx file, then we will use the private key from the pfx file. + */ + + if (Path.GetExtension(Cert).EndsWith("pfx", StringComparison.OrdinalIgnoreCase)) + { + //Create from pfx file including private key + cert = X509Certificate.CreateFromCertFile(Cert); + } + else + { + Validate.EnsureNotNull(PrivKey, "TLS Private Key is required ssl is enabled"); + Validate.FileExists(PrivKey); + + /* + * Attempt to capture the private key password. This will wrap the + * string in a private string instance, and setting the value to true + * will ensure the password memory is wiped when this function returns + */ + using PrivateString? password = PrivateString.ToPrivateString(PrivKeyPassword, true); + + //Load the cert and decrypt with password if set + using X509Certificate2 cert2 = password == null ? X509Certificate2.CreateFromPemFile(Cert, PrivKey) + : X509Certificate2.CreateFromEncryptedPemFile(Cert, password.ToReadOnlySpan(), PrivKey); + + /* + * Workaround for a silly Windows SecureChannel module bug for parsing + * X509Certificate2 from pem cert and private key files. + * + * Must export into pkcs12 format then create a new X509Certificate2 from the + * exported bytes. + */ + + //Copy the cert in pkcs12 format + byte[] pkcs = cert2.Export(X509ContentType.Pkcs12); + cert = new X509Certificate2(pkcs); + MemoryUtil.InitializeBlock(pkcs); + } + + return cert; + } + + /// <summary> + /// Builds a deterministic hash-code base on the configuration state. + /// </summary> + /// <returns>The hash-code that represents the current instance</returns> + public override int GetHashCode() => HashCode.Combine(Address, Port); + + public override bool Equals(object? obj) => obj is TransportInterface iface && GetHashCode() == iface.GetHashCode(); + + public override string ToString() => $"[{Address}:{Port}]"; + + } +} diff --git a/apps/VNLib.WebServer/src/Config/Model/VirtualHostServerConfig.cs b/apps/VNLib.WebServer/src/Config/Model/VirtualHostServerConfig.cs new file mode 100644 index 0000000..5734bd3 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Model/VirtualHostServerConfig.cs @@ -0,0 +1,98 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: VirtualHostServerConfig.cs +* +* VirtualHostServerConfig.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Linq; +using System.Collections.Generic; +using System.Text.Json.Serialization; + +namespace VNLib.WebServer.Config.Model +{ + internal sealed class VirtualHostServerConfig + { + [JsonPropertyName("trace")] + public bool RequestTrace { get; set; } = false; + + [JsonPropertyName("force_port_check")] + public bool ForcePortCheck { get; set; } = false; + + [JsonPropertyName("benchmark")] + public BenchmarkConfig? Benchmark { get; set; } + + [JsonPropertyName("interfaces")] + public TransportInterface[] Interfaces { get; set; } = Array.Empty<TransportInterface>(); + + [JsonPropertyName("hostnames")] + public string[]? Hostnames { get; set; } = Array.Empty<string>(); + + [JsonPropertyName("hostname")] + public string? Hostname + { + get => Hostnames?.FirstOrDefault(); + set + { + if (value != null) + { + Hostnames = [value]; + } + } + } + + [JsonPropertyName("path")] + public string? DirPath { get; set; } = string.Empty; + + [JsonPropertyName("downstream_servers")] + public string[] DownstreamServers { get; set; } = Array.Empty<string>(); + + [JsonPropertyName("whitelist")] + public string[]? Whitelist { get; set; } + + [JsonPropertyName("blacklist")] + public string[]? Blacklist { get; set; } + + [JsonPropertyName("deny_extensions")] + public string[]? DenyExtensions { get; set; } + + [JsonPropertyName("default_files")] + public string[]? DefaultFiles { get; set; } + + [JsonPropertyName("headers")] + public Dictionary<string, string> Headers { get; set; } = []; + + [JsonPropertyName("cors")] + public CorsSecurityConfig Cors { get; set; } = new(); + + [JsonPropertyName("error_files")] + public ErrorFileConfig[] ErrorFiles { get; set; } = Array.Empty<ErrorFileConfig>(); + + [JsonPropertyName("cache_default_sec")] + public int CacheDefaultTimeSeconds { get; set; } = 0; + + [JsonPropertyName("path_filter")] + public string? PathFilter { get; set; } + + [JsonPropertyName("max_execution_time_ms")] + public int MaxExecutionTimeMs { get; set; } = 20000; + } +} diff --git a/apps/VNLib.WebServer/src/Config/ServerConfigurationException.cs b/apps/VNLib.WebServer/src/Config/ServerConfigurationException.cs new file mode 100644 index 0000000..39c18e5 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/ServerConfigurationException.cs @@ -0,0 +1,40 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ServerConfigurationException.cs +* +* ServerConfigurationException.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; + +namespace VNLib.WebServer.Config +{ + public class ServerConfigurationException : Exception + { + public ServerConfigurationException() + { } + + public ServerConfigurationException(string? message) : base(message) + { } + + public ServerConfigurationException(string? message, Exception? innerException) : base(message, innerException) + { } + } +} diff --git a/apps/VNLib.WebServer/src/Config/Validate.cs b/apps/VNLib.WebServer/src/Config/Validate.cs new file mode 100644 index 0000000..773d787 --- /dev/null +++ b/apps/VNLib.WebServer/src/Config/Validate.cs @@ -0,0 +1,113 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: Validate.cs +* +* Validate.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Net; +using System.Diagnostics.CodeAnalysis; +using System.Runtime.CompilerServices; + +using VNLib.Utils.IO; + +namespace VNLib.WebServer.Config +{ + internal static class Validate + { + [DoesNotReturn] + public static void EnsureNotNull<T>(T? obj, string message) where T : class + { + if (obj is null) + { + throw new ServerConfigurationException(message); + } + + if (obj is string s && string.IsNullOrWhiteSpace(s)) + { + throw new ServerConfigurationException(message); + } + } + + public static void Assert([DoesNotReturnIf(false)] bool condition, string message) + { + if (!condition) + { + throw new ServerConfigurationException(message); + } + } + + public static void EnsureValidIp(string? address, string message) + { + if (!IPAddress.TryParse(address, out _)) + { + throw new ServerConfigurationException(message); + } + } + + public static void EnsureNotEqual<T>(T a, T b, string message) + { + if (a is null || b is null) + { + throw new ServerConfigurationException(message); + } + + if (a.Equals(b)) + { + throw new ServerConfigurationException(message); + } + } + + public static void EnsureRangeEx(ulong value, ulong min, ulong max, string message) + { + if (value < min || value > max) + { + throw new ServerConfigurationException(message); + } + } + + public static void EnsureRangeEx(long value, long min, long max, string message) + { + if (value < min || value > max) + { + throw new ServerConfigurationException(message); + } + } + + public static void EnsureRange(ulong value, ulong min, ulong max, [CallerArgumentExpression(nameof(value))] string? paramName = null) + { + EnsureRangeEx(value, min, max, $"Value for {paramName} must be between {min} and {max}. Value: {value}"); + } + + public static void EnsureRange(long value, long min, long max, [CallerArgumentExpression(nameof(value))] string? paramName = null) + { + EnsureRangeEx(value, min, max, $"Value for {paramName} must be between {min} and {max}. Value: {value}"); + } + + public static void FileExists(string path) + { + if (!FileOperations.FileExists(path)) + { + throw new ServerConfigurationException($"Required file: {path} not found"); + } + } + } +} diff --git a/apps/VNLib.WebServer/src/Entry.cs b/apps/VNLib.WebServer/src/Entry.cs new file mode 100644 index 0000000..1ce660d --- /dev/null +++ b/apps/VNLib.WebServer/src/Entry.cs @@ -0,0 +1,335 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: Entry.cs +* +* Entry.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Text; +using System.Text.Json; +using System.Threading; +using System.Net.Sockets; +using System.Runtime.ExceptionServices; + +using VNLib.Utils.IO; +using VNLib.Utils.Memory; +using VNLib.Utils.Logging; +using VNLib.Hashing; +using VNLib.Hashing.Native.MonoCypher; + +using VNLib.WebServer.Config; +using VNLib.WebServer.Bootstrap; +using VNLib.WebServer.RuntimeLoading; + +namespace VNLib.WebServer +{ + + static class Entry + { + const string STARTUP_MESSAGE = +@"VNLib.Webserver - runtime host Copyright (C) Vaughn Nugent +This program comes with ABSOLUTELY NO WARRANTY. +Licensing for this software and other libraries can be found at https://www.vaughnnugent.com/resources/software +Starting... +"; + + private static readonly DirectoryInfo EXE_DIR = new(Environment.CurrentDirectory); + + private const string DEFAULT_CONFIG_PATH = "config.json"; + internal const string TCP_CONF_PROP_NAME = "tcp"; + internal const string LOAD_DEFAULT_HOSTNAME_VALUE = "[system]"; + internal const string PLUGINS_CONFIG_PROP_NAME = "plugins"; + + + static int Main(string[] args) + { + ProcessArguments procArgs = new(args); + + //Print the help menu + if (args.Length == 0 || procArgs.HasArgument("-h") || procArgs.HasArgument("--help")) + { + PrintHelpMenu(); + return 0; + } + + Console.WriteLine(STARTUP_MESSAGE); + + //Init log config builder + ServerLogBuilder logBuilder = new(); + logBuilder.BuildForConsole(procArgs); + + //try to load the json configuration file + IServerConfig? config = LoadConfig(procArgs); + if (config is null) + { + logBuilder.AppLogConfig.CreateLogger().Error("No configuration file was found"); + return -1; + } + + //Build logs from config + logBuilder.BuildFromConfig(config.GetDocumentRoot()); + + //Create the logger + using ServerLogger logger = logBuilder.GetLogger(); + + //Dump config to console + if (procArgs.HasArgument("--dump-config")) + { + DumpConfig(config.GetDocumentRoot(), logger); + } + + //Setup the app-domain listener + InitAppDomainListener(procArgs, logger.AppLog); + +#if !DEBUG + if (procArgs.LogHttp) + { + logger.AppLog.Warn("HTTP Logging is only enabled in builds compiled with DEBUG symbols"); + } +#endif + + if (procArgs.ZeroAllocations && !MemoryUtil.Shared.CreationFlags.HasFlag(HeapCreation.GlobalZero)) + { + logger.AppLog.Debug("Zero allocation flag was set, but the shared heap was not created with the GlobalZero flag, consider enabling zero allocations globally"); + } + + using WebserverBase server = GetWebserver(logger, config, procArgs); + + try + { + logger.AppLog.Information("Building service stack, populating service domain..."); + + server.Configure(); + } + catch (ServerConfigurationException sce) when (sce.InnerException is not null) + { + logger.AppLog.Fatal("Failed to configure server. Reason: {sce}", sce.InnerException.Message); + return -1; + } + catch (ServerConfigurationException sce) + { + logger.AppLog.Fatal("Failed to configure server. Reason: {sce}", sce.Message); + return -1; + } + catch (Exception ex) when (ex.InnerException is ServerConfigurationException sce) + { + logger.AppLog.Fatal("Failed to configure server. Reason: {sce}", sce.Message); + return -1; + } + catch (Exception ex) + { + logger.AppLog.Fatal(ex, "Failed to configure server"); + return -1; + } + + logger.AppLog.Verbose("Server configuration stage complete"); + + using ManualResetEvent ShutdownEvent = new(false); + + try + { + logger.AppLog.Information("Starting services..."); + + server.Start(); + + logger.AppLog.Information("Service stack started, servers are listening."); + + //Register console cancel to cause cleanup + Console.CancelKeyPress += (object? sender, ConsoleCancelEventArgs e) => + { + e.Cancel = true; + ShutdownEvent.Set(); + }; + + /* + * Optional background thread to listen for commands on stdin which + * can also request a server shutdown. + * + * The loop runs in a background thread and will not block the main thread + * The loop can request a server shutdown by setting the shutdown event + */ + + if (!procArgs.HasArgument("--input-off")) + { + CommandListener cmdLoop = new(ShutdownEvent, server, logger.AppLog); + + Thread consoleListener = new(() => cmdLoop.ListenForCommands(Console.In, Console.Out, name: "stdin")) + { + IsBackground = true + }; + + consoleListener.Start(); + } + + logger.AppLog.Information("Main thread waiting for exit signal, press ctrl + c to exit"); + + //Wait for user signal to exit + ShutdownEvent.WaitOne(); + + logger.AppLog.Information("Stopping service stack"); + + server.Stop(); + + //Wait for all plugins to unload and cleanup (temporary) + Thread.Sleep(500); + + return 0; + } + catch (SocketException se) when (se.SocketErrorCode == SocketError.AddressAlreadyInUse) + { + logger.AppLog.Fatal("Failed to start servers, address already in use"); + return (int)se.SocketErrorCode; + } + catch (SocketException se) + { + logger.AppLog.Fatal(se, "Failed to start servers due to a socket exception"); + return (int)se.SocketErrorCode; + } + catch (Exception ex) + { + logger.AppLog.Fatal(ex, "Failed to start web servers"); + } + + return -1; + } + + static void PrintHelpMenu() + { + const string TEMPLATE = +@$" + VNLib.Webserver Copyright (C) 2024 Vaughn Nugent + + A high-performance, cross-platform, single process, reference webserver built on the .NET 8.0 Core runtime. + + Option flags: + --config <path> - Specifies the path to the configuration file (relative or absolute) + --input-off - Disables the STDIN listener, no runtime commands will be processed + --inline-scheduler - Enables inline scheduling for TCP transport IO processing (not available when using TLS) + --no-plugins - Disables loading of dynamic plugins + --log-http - Enables logging of HTTP request and response headers to the system logger (debug builds only) + --log-transport - Enables logging of transport events to the system logger (debug builds only) + --dump-config - Dumps the JSON configuration to the console during loading + --compression-off - Disables dynamic response compression + --zero-alloc - Forces all http/tcp memory pool allocations to be zeroed before use (reduced performance) + --sequential-load - Loads all plugins sequentially (default is concurrently) + --no-reuse-socket - Disables socket reuse for TCP connections (Windows only) + --reuse-address - Enables address reuse for TCP connections + -h, --help - Prints this help menu + -t, --threads <num> - Specifies the number of socket accept threads. Defaults to processor count + -s, --silent - Disables all console logging + -v, --verbose - Enables verbose logging + -d, --debug - Enables debug logging for the process and all plugins + -vv - Enables very verbose logging (attaches listeners for app-domain events and logs them to the output) + + Your configuration file must be a JSON or YAML encoded file and be readable to the process. You may consider keeping it in a safe + location outside the application and only readable to this process. + + You should disable hot-reload for production environments, for security and performance reasons. + + You may consider using the --input-off flag to disable STDIN listening for production environments for security reasons. + + Optional environment variables: + {MemoryUtil.SHARED_HEAP_FILE_PATH} - Specifies the path to the native heap allocator library + {MemoryUtil.SHARED_HEAP_ENABLE_DIAGNOISTICS_ENV} - Enables heap diagnostics for the shared heap 1 = enabled, 0 = disabled + {MemoryUtil.SHARED_HEAP_GLOBAL_ZERO} - Enables zeroing of all allocations from the shared heap 1 = enabled, 0 = disabled + {MemoryUtil.SHARED_HEAP_RAW_FLAGS} - Raw flags to pass to the shared heap allocator's HeapCreate function, hexadeciaml encoded + {VnArgon2.ARGON2_LIB_ENVIRONMENT_VAR_NAME} - Specifies the path to the Argon2 native library + {MonoCypherLibrary.MONOCYPHER_LIB_ENVIRONMENT_VAR_NAME} - Specifies the path to the Monocypher native library + + Usage: + VNLib.Webserver --config <path> ... (other options) #Starts the server from the configuration (basic usage) + +"; + Console.WriteLine(TEMPLATE); + } + + #region config + + /// <summary> + /// Initializes the configuration DOM from the specified cmd args + /// or the default configuration path + /// </summary> + /// <param name="args">The command-line-arguments</param> + /// <returns>A new <see cref="JsonDocument"/> that contains the application configuration</returns> + private static IServerConfig? LoadConfig(ProcessArguments args) + { + //Get the config path or default config + string configPath = args.GetArgument("--config") ?? Path.Combine(EXE_DIR.FullName, DEFAULT_CONFIG_PATH); + + return JsonServerConfig.FromFile(configPath); + } + + private static WebserverBase GetWebserver(ServerLogger logger, IServerConfig config, ProcessArguments procArgs) + { + logger.AppLog.Information("Configuring production webserver"); + return new ReleaseWebserver(logger, config, procArgs); + } + + private static void DumpConfig(JsonElement doc, ServerLogger logger) + { + //Dump the config to the console + using VnMemoryStream ms = new(); + using (Utf8JsonWriter writer = new(ms, new() { Indented = true })) + { + doc.WriteTo(writer); + } + + string json = Encoding.UTF8.GetString(ms.AsSpan()); + logger.AppLog.Information("Dumping configuration to console...\n{c}", json); + } + + #endregion + + private static void InitAppDomainListener(ProcessArguments args, ILogProvider log) + { + AppDomain currentDomain = AppDomain.CurrentDomain; + currentDomain.UnhandledException += delegate (object sender, UnhandledExceptionEventArgs e) + { + log.Fatal("UNHANDLED APPDOMAIN EXCEPTION \n {e}", e); + }; + //If double verbose is specified, log app-domain messages + if (args.DoubleVerbose) + { + log.Verbose("Double verbose mode enabled, registering app-domain listeners"); + + currentDomain.FirstChanceException += delegate (object? sender, FirstChanceExceptionEventArgs e) + { + log.Verbose(e.Exception, "Exception occured in app-domain "); + }; + currentDomain.AssemblyLoad += delegate (object? sender, AssemblyLoadEventArgs args) + { + log.Verbose( + "Assembly loaded {asm} to appdomain {domain} from\n{location}", + args.LoadedAssembly.FullName, + currentDomain.FriendlyName, + args.LoadedAssembly.Location + ); + }; + currentDomain.DomainUnload += delegate (object? sender, EventArgs e) + { + log.Verbose("Domain {domain} unloaded", currentDomain.FriendlyName); + }; + } + } + + } +} diff --git a/apps/VNLib.WebServer/src/MemoryPoolManager.cs b/apps/VNLib.WebServer/src/MemoryPoolManager.cs new file mode 100644 index 0000000..577bdb2 --- /dev/null +++ b/apps/VNLib.WebServer/src/MemoryPoolManager.cs @@ -0,0 +1,137 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: MemoryPoolManager.cs +* +* MemoryPoolManager.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Buffers; +using System.Diagnostics; +using System.Runtime.CompilerServices; + +using VNLib.Utils.Memory; +using VNLib.Net.Http; + +namespace VNLib.WebServer +{ + /// <summary> + /// recovers a memory pool for the TCP server to alloc buffers from + /// </summary> + internal static class MemoryPoolManager + { + /// <summary> + /// Gets an unmanaged memory pool provider for the TCP server to alloc buffers from + /// </summary> + /// <returns>The memory pool</returns> + public static MemoryPool<byte> GetTcpPool(bool zeroOnAlloc) => new HttpMemoryPool(zeroOnAlloc); + + /// <summary> + /// Gets a memory pool provider for the HTTP server to alloc buffers from + /// </summary> + /// <returns>The http server memory pool</returns> + public static IHttpMemoryPool GetHttpPool(bool zeroOnAlloc) => new HttpMemoryPool(zeroOnAlloc); + + /* + * Fun little umnanaged memory pool that allows for allocating blocks + * with fast pointer access and zero cost pinning + * + * All blocks are allocated to the nearest page size + */ + + internal sealed class HttpMemoryPool(bool zeroOnAlloc) : MemoryPool<byte>, IHttpMemoryPool + { + //Avoid the shared getter on every alloc call + private readonly IUnmangedHeap _heap = MemoryUtil.Shared; + + ///<inheritdoc/> + public override int MaxBufferSize { get; } = int.MaxValue; + + ///<inheritdoc/> + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public IMemoryOwner<byte> AllocateBufferForContext(int bufferSize) => Rent(bufferSize); + + ///<inheritdoc/> + public IResizeableMemoryHandle<T> AllocFormDataBuffer<T>(int initialSize) where T : unmanaged + { + return MemoryUtil.SafeAllocNearestPage<T>(_heap, initialSize, zeroOnAlloc); + } + + ///<inheritdoc/> + public override IMemoryOwner<byte> Rent(int minBufferSize = -1) + { + nint initSize = MemoryUtil.NearestPage(minBufferSize); + return new UnsafeMemoryManager(_heap, (nuint)initSize, zeroOnAlloc); + } + + ///<inheritdoc/> + protected override void Dispose(bool disposing) + { } + + sealed class UnsafeMemoryManager(IUnmangedHeap heap, nuint bufferSize, bool zero) : MemoryManager<byte> + { + + private nint _pointer = heap.Alloc(bufferSize, sizeof(byte), zero); + private int _size = (int)bufferSize; + + ///<inheritdoc/> + public override Span<byte> GetSpan() + { + Debug.Assert(_pointer != nint.Zero, "Pointer to memory block is null, was not allocated properly or was released"); + + return MemoryUtil.GetSpan<byte>(_pointer, _size); + } + + ///<inheritdoc/> + public override MemoryHandle Pin(int elementIndex = 0) + { + //Guard + ArgumentOutOfRangeException.ThrowIfNegative(elementIndex); + ArgumentOutOfRangeException.ThrowIfGreaterThanOrEqual(elementIndex, _size); + + Debug.Assert(_pointer != nint.Zero, "Pointer to memory block is null, was not allocated properly or was released"); + + //Get pointer offset from index + nint offset = nint.Add(_pointer, elementIndex); + + //Return handle at offser + return MemoryUtil.GetMemoryHandleFromPointer(offset, pinnable: this); + } + + //No-op + public override void Unpin() + { } + + protected override void Dispose(bool disposing) + { + Debug.Assert(_pointer != nint.Zero, "Pointer to memory block is null, was not allocated properly"); + + bool freed = heap.Free(ref _pointer); + + //Free the memory, should also zero the pointer + Debug.Assert(freed, "Failed to free an allocated block"); + + //Set size to 0 + _size = 0; + } + } + } + } +} diff --git a/apps/VNLib.WebServer/src/Middlewares/BenchmarkMiddleware.cs b/apps/VNLib.WebServer/src/Middlewares/BenchmarkMiddleware.cs new file mode 100644 index 0000000..2e2020c --- /dev/null +++ b/apps/VNLib.WebServer/src/Middlewares/BenchmarkMiddleware.cs @@ -0,0 +1,102 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: BenchmarkMiddleware.cs +* +* BenchmarkMiddleware.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + + +using System; +using System.Net; +using System.Buffers; +using System.Diagnostics; +using System.Threading.Tasks; +using System.Security.Cryptography; + +using VNLib.Utils.Memory; +using VNLib.Utils.Extensions; +using VNLib.Net.Http; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Middleware; + +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer.Middlewares +{ + /* + * This is a cheatsy little syntethic benchmark middleware that will + * return a fixed size memory response for every request to simulate + * a file response for synthetic benchmarking. + * + * The buffer may optionally be filled with random data to put a + * load on the compressor instead of a zero filled buffer + */ + + internal sealed class BenchmarkMiddleware(BenchmarkConfig config) : IHttpMiddleware + { + private readonly MemoryManager<byte> data = AllocBuffer(config.Size, config.Random); + + public ValueTask<FileProcessArgs> ProcessAsync(HttpEntity entity) + { + entity.CloseResponse( + HttpStatusCode.OK, + ContentType.Binary, + new BenchmarkResponseData(data.Memory) + ); + + return ValueTask.FromResult(FileProcessArgs.VirtualSkip); + } + + private static MemoryManager<byte> AllocBuffer(int size, bool random) + { + /* + * Even though this is testing, the buffer is zeroed to avoid leaking + * any data that may be in heap memory after allocation. + */ + MemoryManager<byte> man = MemoryUtil.Shared.DirectAlloc<byte>(size, true); + + if (random) + { + RandomNumberGenerator.Fill(man.GetSpan()); + } + + return man; + } + + private sealed class BenchmarkResponseData(ReadOnlyMemory<byte> data) : IMemoryResponseReader + { + int read; + readonly int len = data.Length; + + public int Remaining => len - read; + + public void Advance(int written) + { + read += written; + Debug.Assert(Remaining >= 0); + } + + public void Close() + { } + + public ReadOnlyMemory<byte> GetMemory() => data.Slice(read, Remaining); + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Middlewares/CORSMiddleware.cs b/apps/VNLib.WebServer/src/Middlewares/CORSMiddleware.cs new file mode 100644 index 0000000..08b7cee --- /dev/null +++ b/apps/VNLib.WebServer/src/Middlewares/CORSMiddleware.cs @@ -0,0 +1,182 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: CORSMiddleware.cs +* +* CORSMiddleware.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Net; +using System.Threading.Tasks; +using System.Collections.Frozen; + +using VNLib.Net.Http; +using VNLib.Utils.Logging; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Sessions; +using VNLib.Plugins.Essentials.Extensions; +using VNLib.Plugins.Essentials.Middleware; + +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer.Middlewares +{ + + /// <summary> + /// Adds HTTP CORS protection to http servers + /// </summary> + /// <param name="Log"></param> + /// <param name="VirtualHostOptions"></param> + [MiddlewareImpl(MiddlewareImplOptions.SecurityCritical)] + internal sealed class CORSMiddleware(ILogProvider Log, CorsSecurityConfig secConfig) : IHttpMiddleware + { + private readonly FrozenSet<string> _corsAuthority = secConfig.AllowedCorsAuthority.ToFrozenSet(StringComparer.OrdinalIgnoreCase); + + public ValueTask<FileProcessArgs> ProcessAsync(HttpEntity entity) + { + //Check coors enabled + bool isCors = entity.Server.IsCors(); + bool isCrossSite = entity.Server.IsCrossSite(); + + /* + * Deny/allow cross site/cors requests at the site-level + */ + if (!secConfig.DenyCorsCons) + { + //Confirm the origin is allowed during cors connections + if (entity.Server.CrossOrigin && _corsAuthority.Count > 0) + { + //If the authority is not allowed, deny the connection + if (!_corsAuthority.Contains(entity.Server.Origin!.Authority)) + { + Log.Debug("Denied a connection from a cross-origin site {s}, the origin was not whitelisted", entity.Server.Origin); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + } + + if (isCors) + { + //set the allow credentials header + entity.Server.Headers["Access-Control-Allow-Credentials"] = "true"; + + //If cross site flag is set, or the connection has cross origin flag set, set explicit origin + if (entity.Server.CrossOrigin || isCrossSite && entity.Server.Origin != null) + { + entity.Server.Headers["Access-Control-Allow-Origin"] = $"{entity.Server.RequestUri.Scheme}://{entity.Server.Origin!.Authority}"; + //Add origin to the response vary header when setting cors origin + entity.Server.Headers.Append(HttpResponseHeader.Vary, "Origin"); + } + } + + //Add sec vary headers for cors enabled sites + entity.Server.Headers.Append(HttpResponseHeader.Vary, "Sec-Fetch-Dest,Sec-Fetch-Mode,Sec-Fetch-Site"); + } + else if (isCors | isCrossSite) + { + Log.Verbose("Denied a cross-site/cors request from {con} because this site does not allow cross-site/cors requests", entity.TrustedRemoteIp); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + + //If user-navigation is set and method is get, make sure it does not contain object/embed + if (entity.Server.IsNavigation() && entity.Server.Method == HttpMethod.GET) + { + string? dest = entity.Server.Headers["sec-fetch-dest"]; + if (dest != null && (dest.Contains("object", StringComparison.OrdinalIgnoreCase) || dest.Contains("embed", StringComparison.OrdinalIgnoreCase))) + { + Log.Debug("Denied a browser navigation request from {con} because it contained an object/embed", entity.TrustedRemoteIp); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + } + + //If the connection is a cross-site, then an origin header must be supplied + if (isCrossSite && entity.Server.Origin is null) + { + Log.Debug("Denied cross-site request because origin header was not supplied"); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + + //If same origin is supplied, enforce origin header on post/options/put/patch + if (string.Equals("same-origin", entity.Server.Headers["Sec-Fetch-Site"], StringComparison.OrdinalIgnoreCase)) + { + //If method is not get/head, then origin is required + if ((entity.Server.Method & (HttpMethod.GET | HttpMethod.HEAD)) == 0 && entity.Server.Origin == null) + { + Log.Debug("Denied same-origin POST/PUT... request because origin header was not supplied"); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + } + + if(!IsSessionSecured(entity)) + { + return ValueTask.FromResult(FileProcessArgs.Deny); + } + + return ValueTask.FromResult(FileProcessArgs.Continue); + } + + private bool IsSessionSecured(HttpEntity entity) + { + ref readonly SessionInfo session = ref entity.Session; + + /* + * When sessions are created for connections that come from a different + * origin, their origin is stored for later. + * + * If the session was created from a different origin or the current connection + * is cross origin, then the origin must be allowed by the configuration + */ + + //No session loaded, nothing to check + if (!session.IsSet) + { + return true; + } + + if (entity.Server.Origin is null) + { + return true; + } + + if (session.IsNew || session.SessionType != SessionType.Web) + { + return true; + } + + bool sameOrigin = string.Equals( + entity.Server.Origin.Authority, + session.SpecifiedOrigin?.Authority, + StringComparison.OrdinalIgnoreCase + ); + + if (sameOrigin || _corsAuthority.Contains(entity.Server.Origin.Authority)) + { + return true; + } + + Log.Debug("Denied connection from {0} because the user's origin {org} changed to {other} and is not whitelisted.", + entity.TrustedRemoteIp, + session.SpecifiedOrigin?.Authority, + entity.Server.Origin.Authority + ); + + return false; + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Middlewares/ConnectionLogMiddleware.cs b/apps/VNLib.WebServer/src/Middlewares/ConnectionLogMiddleware.cs new file mode 100644 index 0000000..939fd0d --- /dev/null +++ b/apps/VNLib.WebServer/src/Middlewares/ConnectionLogMiddleware.cs @@ -0,0 +1,108 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ConnectionLogMiddleware.cs +* +* ConnectionLogMiddleware.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +/* + * Provides an Nginx-style log of incoming connections. + */ + +using System.Threading.Tasks; +using System.Security.Authentication; + +using VNLib.Net.Http; +using VNLib.Utils.Logging; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Middleware; + +namespace VNLib.WebServer.Middlewares +{ + internal sealed class ConnectionLogMiddleware(ILogProvider Log) : IHttpMiddleware + { + const string template = @"{ip} - {usr} [{local_time}] {tls} '{method} {url} {http_version}' {hostname} '{refer}' '{user_agent}' '{forwarded_for}'"; + + ///<inheritdoc/> + public ValueTask<FileProcessArgs> ProcessAsync(HttpEntity entity) + { + if (Log.IsEnabled(LogLevel.Information)) + { + string userId = string.Empty; + if (entity.Session.IsSet) + { + userId = entity.Session.UserID; + } + + Log.Information(template, + entity.TrustedRemoteIp, + userId, + entity.RequestedTimeUtc.ToLocalTime().ToString("dd/MMM/yyyy:HH:mm:ss zzz", null), + GetTlsInfo(entity), + entity.Server.Method, + entity.Server.RequestUri.PathAndQuery, + GetProtocolVersionString(entity), + entity.RequestedRoot.Hostname, + entity.Server.Referer, + entity.Server.UserAgent, + entity.Server.Headers["X-Forwarded-For"] ?? string.Empty + ); + } + + return ValueTask.FromResult(FileProcessArgs.Continue); + } + + static string GetProtocolVersionString(HttpEntity entity) + { + return entity.Server.ProtocolVersion switch + { + HttpVersion.Http09 => "HTTP/0.9", + HttpVersion.Http1 => "HTTP/1.0", + HttpVersion.Http11 => "HTTP/1.1", + HttpVersion.Http2 => "HTTP/2.0", + HttpVersion.Http3 => "HTTP/3.0", + _ => "HTTP/1.1" + }; + } + + static string GetTlsInfo(HttpEntity entity) + { + ref readonly TransportSecurityInfo? secInfo = ref entity.Server.GetTransportSecurityInfo(); + + if(!secInfo.HasValue) + { + return string.Empty; + } + +#pragma warning disable CA5398, CA5397, SYSLIB0039 // Avoid hardcoded SslProtocols values + + return secInfo.Value.SslProtocol switch + { + SslProtocols.Tls => "TLSv1.0", + SslProtocols.Tls11 => "TLSv1.1", + SslProtocols.Tls12 => "TLSv1.2", + SslProtocols.Tls13 => "TLSv1.3", + _ => "Unknown" + }; + +#pragma warning restore CA5397, CA5398, SYSLIB0039 // Do not use deprecated SslProtocols values + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Middlewares/IpBlacklistMiddleware.cs b/apps/VNLib.WebServer/src/Middlewares/IpBlacklistMiddleware.cs new file mode 100644 index 0000000..5959a95 --- /dev/null +++ b/apps/VNLib.WebServer/src/Middlewares/IpBlacklistMiddleware.cs @@ -0,0 +1,49 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: WhitelistMiddleware.cs +* +* WhitelistMiddleware.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Net; +using System.Threading.Tasks; +using System.Collections.Frozen; + +using VNLib.Utils.Logging; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Middleware; + +namespace VNLib.WebServer.Middlewares +{ + [MiddlewareImpl(MiddlewareImplOptions.SecurityCritical)] + internal sealed class IpBlacklistMiddleware(ILogProvider Log, FrozenSet<IPAddress> Blacklist) : IHttpMiddleware + { + public ValueTask<FileProcessArgs> ProcessAsync(HttpEntity entity) + { + if (Blacklist.Contains(entity.TrustedRemoteIp)) + { + Log.Verbose("Client {ip} is blacklisted, blocked", entity.TrustedRemoteIp); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + + return ValueTask.FromResult(FileProcessArgs.Continue); + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Middlewares/IpWhitelistMiddleware.cs b/apps/VNLib.WebServer/src/Middlewares/IpWhitelistMiddleware.cs new file mode 100644 index 0000000..bda754d --- /dev/null +++ b/apps/VNLib.WebServer/src/Middlewares/IpWhitelistMiddleware.cs @@ -0,0 +1,53 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: WhitelistMiddleware.cs +* +* WhitelistMiddleware.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Net; +using System.Threading.Tasks; +using System.Collections.Frozen; + +using VNLib.Utils.Logging; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Middleware; + +namespace VNLib.WebServer.Middlewares +{ + /* + * Middelware that matches clients real ip addresses against a whitelist + * and blocks them if they are not on the list + */ + [MiddlewareImpl(MiddlewareImplOptions.SecurityCritical)] + internal sealed class IpWhitelistMiddleware(ILogProvider Log, FrozenSet<IPAddress> WhiteList) : IHttpMiddleware + { + public ValueTask<FileProcessArgs> ProcessAsync(HttpEntity entity) + { + if (!WhiteList.Contains(entity.TrustedRemoteIp)) + { + Log.Verbose("Client {ip} is not whitelisted, blocked", entity.TrustedRemoteIp); + return ValueTask.FromResult(FileProcessArgs.Deny); + } + + return ValueTask.FromResult(FileProcessArgs.Continue); + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Middlewares/MainServerMiddlware.cs b/apps/VNLib.WebServer/src/Middlewares/MainServerMiddlware.cs new file mode 100644 index 0000000..da7eb3d --- /dev/null +++ b/apps/VNLib.WebServer/src/Middlewares/MainServerMiddlware.cs @@ -0,0 +1,100 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: MainServerMiddlware.cs +* +* MainServerMiddlware.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Net; +using System.Threading.Tasks; +using System.Collections.Generic; + +using VNLib.Net.Http; +using VNLib.Utils.Logging; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Extensions; +using VNLib.Plugins.Essentials.Middleware; + +namespace VNLib.WebServer.Middlewares +{ + + /// <summary> + /// Provides required/essential server functionality as a middelware processor + /// </summary> + /// <param name="Log"></param> + /// <param name="VirtualHostOptions"></param> + internal sealed class MainServerMiddlware(ILogProvider Log, VirtualHostConfig VirtualHostOptions, bool forcePorts) : IHttpMiddleware + { + public ValueTask<FileProcessArgs> ProcessAsync(HttpEntity entity) + { + //Set special server header + VirtualHostOptions.TrySetSpecialHeader(entity.Server, SpecialHeaders.Server); + + //Block websocket requests + if (entity.Server.IsWebSocketRequest) + { + Log.Verbose("Client {ip} made a websocket request", entity.TrustedRemoteIp); + } + + ref readonly TransportSecurityInfo? tlsSecInfo = ref entity.Server.GetTransportSecurityInfo(); + + //Check transport security if set + if (tlsSecInfo.HasValue) + { + + } + + //If not behind upstream server, uri ports and server ports must match + bool enforcePortCheck = !entity.IsBehindDownStreamServer && forcePorts; + + if (enforcePortCheck && !entity.Server.EnpointPortsMatch()) + { + Log.Debug("Connection {ip} received on port {p} but the client host port did not match at {pp}", + entity.TrustedRemoteIp, + entity.Server.LocalEndpoint.Port, + entity.Server.RequestUri.Port + ); + + return ValueTask.FromResult(FileProcessArgs.Deny); + } + + /* + * downstream server will handle the transport security, + * if the connection is not from an downstream server + * and is using transport security then we can specify HSTS + */ + if (entity.IsSecure) + { + VirtualHostOptions.TrySetSpecialHeader(entity.Server, SpecialHeaders.Hsts); + } + + //Add response headers from vh config + for (int i = 0; i < VirtualHostOptions.AdditionalHeaders.Length; i++) + { + //Get and append the client header value + ref KeyValuePair<string, string> header = ref VirtualHostOptions.AdditionalHeaders[i]; + + entity.Server.Headers.Append(header.Key, header.Value); + } + + return ValueTask.FromResult(FileProcessArgs.Continue); + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/Plugins/PluginAssemblyLoader.cs b/apps/VNLib.WebServer/src/Plugins/PluginAssemblyLoader.cs new file mode 100644 index 0000000..b49c782 --- /dev/null +++ b/apps/VNLib.WebServer/src/Plugins/PluginAssemblyLoader.cs @@ -0,0 +1,125 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: PluginAssemblyLoader.cs +* +* PluginAssemblyLoader.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Diagnostics; +using System.Reflection; +using System.Runtime.Loader; + +#if USE_MCMASTER +using McMaster.NETCore.Plugins; +#endif + +using VNLib.Plugins.Runtime; +using VNLib.Utils.Resources; + +namespace VNLib.WebServer.Plugins +{ + + internal static class PluginAsemblyLoading + { + public static IAssemblyLoader Create(IPluginAssemblyLoadConfig config) + { + return config.Unloadable ? new UnloadableAlc(config) : new ImmutableAl(config); + } + + //Immutable assembly loader + internal sealed record class ImmutableAl(IPluginAssemblyLoadConfig Config) : IAssemblyLoader + { + private readonly AssemblyLoadContext ctx = new(Config.AssemblyFile, Config.Unloadable); + private ManagedLibrary ml = null!; + + ///<inheritdoc/> + public Assembly GetAssembly() => ml.Assembly; + + ///<inheritdoc/> + public void Load() => ml = ManagedLibrary.LoadManagedAssembly(Config.AssemblyFile, ctx); + + ///<inheritdoc/> + public void Unload() => Debug.Fail("Unload was called on an immutable assembly loader"); + + public void Dispose() { } + } + + internal sealed record class UnloadableAlc(IPluginAssemblyLoadConfig Config) : IAssemblyLoader + { + +#if USE_MCMASTER + private readonly PluginLoader _loader = new(new(Config.AssemblyFile) + { + PreferSharedTypes = true, + IsUnloadable = Config.Unloadable, + LoadInMemory = Config.Unloadable + }); + + ///<inheritdoc/> + public Assembly GetAssembly() => _loader.LoadDefaultAssembly(); + + ///<inheritdoc/> + public void Load() => _loader.Load(); + + ///<inheritdoc/> + public void Unload() + { + if (Config.Unloadable) + { + //Cleanup old loader, dont invoke GC because runtime will handle it + _loader.Destroy(false); + //ctx.Unload(); + //ml = null!; + + //Init new load context with the same name + //ctx = new AssemblyLoadContext(Config.AssemblyFile, Config.Unloadable); + } + } + + public void Dispose() => Unload(); +#else + + private AssemblyLoadContext ctx = null!; + private ManagedLibrary ml = null!; + + public void Dispose() => Unload(); + + public Assembly GetAssembly() => ml.Assembly; + + public void Load() + { + Debug.Assert(Config.Unloadable, "Assumed unloadable context when using UnloadableAlc"); + + //A new load context is created for each load + ctx = new(Config.AssemblyFile, Config.Unloadable); + ml = ManagedLibrary.LoadManagedAssembly(Config.AssemblyFile, ctx); + } + + public void Unload() + { + ctx.Unload(); + ml = null!; + } +#endif + + } + } +} diff --git a/apps/VNLib.WebServer/src/RuntimeLoading/ProcessArguments.cs b/apps/VNLib.WebServer/src/RuntimeLoading/ProcessArguments.cs new file mode 100644 index 0000000..b6db2e6 --- /dev/null +++ b/apps/VNLib.WebServer/src/RuntimeLoading/ProcessArguments.cs @@ -0,0 +1,40 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ProcessArguments.cs +* +* ProcessArguments.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Collections.Generic; + +using VNLib.Utils; + +namespace VNLib.WebServer.RuntimeLoading +{ + internal sealed class ProcessArguments(IEnumerable<string> args) : ArgumentList(args) + { + public bool Verbose => HasArgument("-v") || HasArgument("--verbose"); + public bool Debug => HasArgument("-d") || HasArgument("--debug"); + public bool Silent => HasArgument("-s") || HasArgument("--silent"); + public bool DoubleVerbose => Verbose && HasArgument("-vv"); + public bool LogHttp => HasArgument("--log-http"); + public bool ZeroAllocations => HasArgument("--zero-alloc"); + } +} diff --git a/apps/VNLib.WebServer/src/RuntimeLoading/ServerLogBuilder.cs b/apps/VNLib.WebServer/src/RuntimeLoading/ServerLogBuilder.cs new file mode 100644 index 0000000..a10a25e --- /dev/null +++ b/apps/VNLib.WebServer/src/RuntimeLoading/ServerLogBuilder.cs @@ -0,0 +1,136 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ServerLogBuilder.cs +* +* ServerLogBuilder.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Text.Json; + +using Serilog; + +using VNLib.WebServer.Config; +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer.RuntimeLoading +{ + internal sealed class ServerLogBuilder + { + public LoggerConfiguration SysLogConfig { get; } = new(); + + public LoggerConfiguration AppLogConfig { get; } = new(); + + public LoggerConfiguration? DebugConfig { get; } + + + public ServerLogBuilder BuildForConsole(ProcessArguments args) + { + InitConsoleLog(args, AppLogConfig, "Application"); + InitConsoleLog(args, SysLogConfig, "System"); + return this; + } + + public ServerLogBuilder BuildFromConfig(JsonElement logEl) + { + if(logEl.TryGetProperty("logs", out logEl)) + { + InitSingleLog(logEl, "app_log", "Application", AppLogConfig); + InitSingleLog(logEl, "sys_log", "System", SysLogConfig); + } + + return this; + } + + public ServerLogger GetLogger() + { + //Return logger + return new ( + new(AppLogConfig), + new(SysLogConfig), + DebugConfig == null ? null : new(DebugConfig) + ); + } + + private static void InitConsoleLog(ProcessArguments args, LoggerConfiguration conf, string logName) + { + //Set verbosity level, defaul to informational + if (args.Verbose) + { + conf.MinimumLevel.Verbose(); + } + else if (args.Debug) + { + conf.MinimumLevel.Debug(); + } + else + { + conf.MinimumLevel.Information(); + } + + //Setup loggers to write to console unless the -s silent arg is set + if (!args.Silent) + { + string template = $"{{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz}} [{{Level:u3}}] {logName} {{Message:lj}}{{NewLine}}{{Exception}}"; + _ = conf.WriteTo.Console(outputTemplate: template); + } + } + + private static void InitSingleLog(JsonElement el, string elPath, string logName, LoggerConfiguration logConfig) + { + if(!el.TryGetProperty(elPath, out el)) + { + return; + } + + LogConfig? conf = el.Deserialize<LogConfig>(); + + if(conf == null || !conf.Enabled) + { + return; + } + + //Default path if the user did not set one or set it to null + conf.Path ??= Path.Combine(Environment.CurrentDirectory, $"{elPath}.txt"); + + conf.Template ??= $"{{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz}} [{{Level:u3}}] {logName} {{Message:lj}}{{NewLine}}{{Exception}}"; + + Validate.EnsureNotNull(conf.Interval, "You must not set a null rolling log interval"); + Validate.EnsureRange(conf.FlushIntervalSeconds, 1, 6000, "flush_sec"); + Validate.EnsureRange(conf.RetainedFiles, 1, 100, "retained_files"); + Validate.EnsureRange(conf.FileSizeLimit, 100, 1000000000, "file_size_limit"); + + //Configure the log file writer + logConfig.WriteTo.File( + path: conf.Path, + buffered: true, + retainedFileCountLimit: conf.RetainedFiles, + formatProvider: null, + fileSizeLimitBytes: conf.FileSizeLimit, + rollingInterval: Enum.Parse<RollingInterval>(conf.Interval, ignoreCase: true), + outputTemplate: conf.Template, + flushToDiskInterval: TimeSpan.FromSeconds(conf.FlushIntervalSeconds) + ); + + //If the log element is not specified in config, do not write log files + } + } +} diff --git a/apps/VNLib.WebServer/src/RuntimeLoading/ServerLogger.cs b/apps/VNLib.WebServer/src/RuntimeLoading/ServerLogger.cs new file mode 100644 index 0000000..7b93bfe --- /dev/null +++ b/apps/VNLib.WebServer/src/RuntimeLoading/ServerLogger.cs @@ -0,0 +1,45 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: ServerLogger.cs +* +* ServerLogger.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using VNLib.Utils; + +namespace VNLib.WebServer.RuntimeLoading +{ + internal sealed class ServerLogger(VLogProvider applog, VLogProvider syslog, VLogProvider? debuglog) : VnDisposeable + { + + public VLogProvider AppLog { get; } = applog; + + public VLogProvider SysLog { get; } = syslog; + + public VLogProvider? DebugLog { get; } = debuglog; + + protected override void Free() + { + AppLog.Dispose(); + SysLog.Dispose(); + DebugLog?.Dispose(); + } + } +} diff --git a/apps/VNLib.WebServer/src/Transport/HostAwareServerSslOptions.cs b/apps/VNLib.WebServer/src/Transport/HostAwareServerSslOptions.cs new file mode 100644 index 0000000..ed97830 --- /dev/null +++ b/apps/VNLib.WebServer/src/Transport/HostAwareServerSslOptions.cs @@ -0,0 +1,108 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: HostAwareServerSslOptions.cs +* +* HostAwareServerSslOptions.cs is part of VNLib.WebServer which is part +* of the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Net.Security; +using System.Collections.Generic; +using System.Security.Authentication; +using System.Security.Cryptography.X509Certificates; + +using VNLib.WebServer.Config; +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer.Transport +{ + + internal sealed class HostAwareServerSslOptions : SslServerAuthenticationOptions + { + //TODO programatically setup ssl protocols, but for now we only use HTTP/1.1 so this can be hard-coded + internal static readonly List<SslApplicationProtocol> SslAppProtocols = new() + { + SslApplicationProtocol.Http11, + //SslApplicationProtocol.Http2, + }; + + private readonly bool _clientCertRequired; + private readonly X509Certificate _cert; + + private readonly SslPolicyErrors _errorLevel; + + public HostAwareServerSslOptions(TransportInterface iFace) + { + ArgumentNullException.ThrowIfNull(iFace); + Validate.Assert(iFace.Ssl, "An interface was selected that does not have SSL enabled. This is likely a bug"); + + _clientCertRequired = iFace.ClientCertRequired; + _cert = iFace.LoadCertificate()!; + + /* + * If client certificates are required, then no policy errors are allowed + * and the certificate must be requested from the user. Otherwise, no errors + * except missing certificates are allowed + */ + _errorLevel = _clientCertRequired + ? SslPolicyErrors.None + : SslPolicyErrors.RemoteCertificateNotAvailable; + + //Set validation callback + RemoteCertificateValidationCallback = OnRemoteCertVerification; + ServerCertificateSelectionCallback = OnGetCertificatForHost; + + ConfigureBaseDefaults(iFace.UseOsCiphers); + } + + private void ConfigureBaseDefaults(bool doNotForceProtocols) + { + //Eventually when HTTP2 is supported, we can select the ssl version to match + ApplicationProtocols = SslAppProtocols; + + AllowRenegotiation = false; + EncryptionPolicy = EncryptionPolicy.RequireEncryption; + + //Allow user to disable forced protocols and let the os decide + EnabledSslProtocols = doNotForceProtocols + ? SslProtocols.None + : SslProtocols.Tls12 | SslProtocols.Tls13; + } + + private bool OnRemoteCertVerification(object sender, X509Certificate? certificate, X509Chain? chain, SslPolicyErrors sslPolicyErrors) + { + /* + * Since certificates are loaded at the interface level, an interface is defined at the virtual host level + * and can only accept one certificate per virtual host. So SNI is not useful here since certificates are + * verified at the interface level. + */ + + return _errorLevel == sslPolicyErrors; + } + + /* + * Callback for getting the certificate from a hostname + * + * Always used the certificate defined at the interface level + */ + private X509Certificate OnGetCertificatForHost(object sender, string? hostName) => _cert; + + } +} diff --git a/apps/VNLib.WebServer/src/Transport/SslTcpTransportContext.cs b/apps/VNLib.WebServer/src/Transport/SslTcpTransportContext.cs new file mode 100644 index 0000000..8e3c2fd --- /dev/null +++ b/apps/VNLib.WebServer/src/Transport/SslTcpTransportContext.cs @@ -0,0 +1,103 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: SslTcpTransportContext.cs +* +* SslTcpTransportContext.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + + +using System.Net.Security; +using System.Threading.Tasks; +using System.Runtime.CompilerServices; + +using VNLib.Net.Http; +using VNLib.Net.Transport.Tcp; + +namespace VNLib.WebServer.Transport +{ + internal sealed class SslTcpTransportContext(ITcpListner server, ITcpConnectionDescriptor descriptor, SslStream stream) + : TcpTransportContext(server, descriptor, stream) + { + private TransportSecurityInfo? _securityInfo; + private readonly SslStream _baseStream = stream; + + ///<inheritdoc/> + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public async override ValueTask CloseConnectionAsync() + { + try + { + //Shutdown the ssl stream before cleaning up the connection + await _baseStream.ShutdownAsync(); + await _connectionStream.DisposeAsync(); + } + finally + { + //Always close the underlying connection + await base.CloseConnectionAsync(); + } + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public override ref readonly TransportSecurityInfo? GetSecurityInfo() + { + //Value has not been loaded yet, so lazy load it + if (!_securityInfo.HasValue) + { + //Create sec info from the ssl stream + GetSecInfo(ref _securityInfo, _baseStream); + } + + return ref _securityInfo; + } + + + //Lazy load sec info + private static void GetSecInfo(ref TransportSecurityInfo? tsi, SslStream ssl) + { + //Build sec info + tsi = new() + { + SslProtocol = ssl.SslProtocol, + HashAlgorithm = ssl.HashAlgorithm, + CipherAlgorithm = ssl.CipherAlgorithm, + + HashStrength = ssl.HashStrength, + CipherStrength = ssl.CipherStrength, + + IsSigned = ssl.IsSigned, + IsEncrypted = ssl.IsEncrypted, + IsAuthenticated = ssl.IsAuthenticated, + IsMutuallyAuthenticated = ssl.IsMutuallyAuthenticated, + CheckCertRevocationStatus = ssl.CheckCertRevocationStatus, + + KeyExchangeStrength = ssl.KeyExchangeStrength, + KeyExchangeAlgorithm = ssl.KeyExchangeAlgorithm, + + LocalCertificate = ssl.LocalCertificate, + RemoteCertificate = ssl.RemoteCertificate, + + TransportContext = ssl.TransportContext, + NegotiatedCipherSuite = ssl.NegotiatedCipherSuite, + NegotiatedApplicationProtocol = ssl.NegotiatedApplicationProtocol, + }; + } + } +} diff --git a/apps/VNLib.WebServer/src/Transport/TcpServerLoader.cs b/apps/VNLib.WebServer/src/Transport/TcpServerLoader.cs new file mode 100644 index 0000000..12ad52f --- /dev/null +++ b/apps/VNLib.WebServer/src/Transport/TcpServerLoader.cs @@ -0,0 +1,221 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: TcpServerLoader.cs +* +* TcpServerLoader.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Linq; +using System.Text.Json; +using System.Net.Sockets; +using System.Collections.Generic; + +using VNLib.Utils.Logging; +using VNLib.Utils.Resources; +using VNLib.Net.Http; +using VNLib.Net.Transport.Tcp; + +using VNLib.WebServer.Config; +using VNLib.WebServer.RuntimeLoading; +using VNLib.WebServer.Config.Model; + +using VNLib.Plugins.Essentials.ServiceStack; +using VNLib.Plugins.Essentials.ServiceStack.Construction; + +namespace VNLib.WebServer.Transport +{ + internal sealed class TcpServerLoader(IServerConfig hostConfig, ProcessArguments args, ILogProvider tcpLogger) + { + const int CacheQuotaDefault = 0; //Disable cache quota by default, allows unlimited cache + + const string TransportLogTemplate = +@"Interface (TCP/IP): {iface} RX: {rx} TX: {tx} TLS: {tls} threads: {threads} max-cons: {max} Keepalive: {keepalive}"; + + private readonly LazyInitializer<TcpConfigJson> _conf = new(() => + { + JsonElement rootElement = hostConfig.GetDocumentRoot(); + + if (rootElement.TryGetProperty(Entry.TCP_CONF_PROP_NAME, out JsonElement tcpEl)) + { + return tcpEl.DeserializeElement<TcpConfigJson>()!; + } + + return new TcpConfigJson(); + }); + + private readonly bool UseInlineScheduler = args.HasArgument("--inline-scheduler"); + private readonly string? ThreadCountArg = args.GetArgument("-t") ?? args.GetArgument("--threads"); + private readonly bool EnableTransportLogging = args.HasArgument("--log-transport"); + private readonly bool NoReuseSocket = args.HasArgument("--no-reuse-socket"); + private readonly bool ReuseAddress = args.HasArgument("--reuse-address"); + + /// <summary> + /// The user confiugred TCP transmission buffer size + /// </summary> + public int TcpTxBufferSize => _conf.Instance.TcpSendBufferSize; + + /// <summary> + /// The user-conifuigred TCP receive buffer size + /// </summary> + public int TcpRxBufferSize => _conf.Instance.TcpRecvBufferSize; + + public HttpTransportMapping[] ReduceBindingsForGroups(IReadOnlyCollection<ServiceGroup> groups) + { + /* + * All transports can be reduced by their endpoints to reduce the number of + * TCP server instances that need to be created. + * + * The following code attempts to reorder the many-to-many set mapping of + * transports to virtual hosts into a set of one-to-many http bingings. + * + * Example: + * + * Virtual hosts Transports + * ex.com 0.0.0.0:80 (no ssl) + * ex.com 0.0.0.0:443 (ssl) (shared) + * + * ex1.com 192.168.1.6:80 (no ssl) + * ex1.com 0.0.0.0:443 (ssl) (shared) + */ + + Dictionary<TransportInterface, ITransportProvider> transportMap = groups + .SelectMany(static g => g.Hosts) + .Select(static c => (VirtualHostConfig)c.UserState!) + .SelectMany(static c => c.Transports) + .DistinctBy(static t => t.GetHashCode()) + .ToDictionary(static k => k, GetProviderForInterface); + + /* + * The following code groups virtual hosts that share the same transport + * interface and creates a new HttpTransportMapping instance for each + * group of shared interfaces, pulling the transport provider from the + * transport map above. + */ + + HttpTransportMapping[] bindings = groups + .SelectMany(static s => s.Hosts) + .SelectMany(static host => + { + //The vhost config is stored as a user-object on the service host + VirtualHostConfig config = (VirtualHostConfig)host.UserState!; + + return config.Transports.Select(iface => new OneToOneHostMapping(host, iface)); + }) + .GroupBy(static m => m.Interface.GetHashCode()) + .Select(otoMap => + { + IServiceHost[] sharedTransportHosts = otoMap + .Select(static m => m.Host) + .ToArray(); + + TransportInterface sharedInterface = otoMap.First().Interface; + + //Find any duplicate hostnames that share the same transport interface and raise validation exception + string[] sharedHostErrors = sharedTransportHosts.GroupBy(static h => h.Processor.Hostname) + .Where(static g => g.Count() > 1) + .Select(duplicteGroup => + { + string hostnames = string.Join(", ", duplicteGroup.Select(h => h.Processor.Hostname)); + + return $"Duplicate hostnames: {hostnames} share the same transport interface {sharedInterface}"; + }) + .ToArray(); + + //If any duplicate hostnames are found, raise a validation exception + if (sharedHostErrors.Length > 0) + { + throw new ServerConfigurationException(string.Join('\n', sharedHostErrors)); + } + + ITransportProvider mappedTransport = transportMap[sharedInterface]; + + return new HttpTransportMapping(sharedTransportHosts, mappedTransport); + }) + .ToArray(); + + return bindings; + } + + sealed record class OneToOneHostMapping(IServiceHost Host, TransportInterface Interface); + + private ITransportProvider GetProviderForInterface(TransportInterface iface) + { + if (!uint.TryParse(ThreadCountArg, out uint threadCount)) + { + threadCount = (uint)Environment.ProcessorCount; + } + + TcpConfigJson baseConfig = _conf.Instance; + baseConfig.ValidateConfig(); + + TCPConfig tcpConf = new() + { + LocalEndPoint = iface.GetEndpoint(), + AcceptThreads = threadCount, + CacheQuota = CacheQuotaDefault, + Log = tcpLogger, + DebugTcpLog = EnableTransportLogging, //Only available in debug logging + BackLog = baseConfig.BackLog, + MaxConnections = baseConfig.MaxConnections, + TcpKeepAliveTime = baseConfig.TcpKeepAliveTime, + KeepaliveInterval = baseConfig.KeepaliveInterval, + MaxRecvBufferData = baseConfig.MaxRecvBufferData, + ReuseSocket = !NoReuseSocket, //Default to always reuse socket if allowed + OnSocketCreated = OnSocketConfiguring, + BufferPool = MemoryPoolManager.GetTcpPool(args.ZeroAllocations) + }; + + //Print warning message, since inline scheduler is an avanced feature + if (iface.Ssl && UseInlineScheduler) + { + tcpLogger.Debug("[WARN]: Inline scheduler is not available on server {server} when using TLS", tcpConf.LocalEndPoint); + } + + tcpLogger.Verbose(TransportLogTemplate, + iface, + baseConfig.TcpRecvBufferSize, + baseConfig.TcpSendBufferSize, + iface.Ssl, + threadCount, + tcpConf.MaxConnections, + tcpConf.BackLog, + tcpConf.TcpKeepAliveTime > 0 ? $"{tcpConf.TcpKeepAliveTime} sec" : "Disabled" + ); + + //Init new tcp server with/without ssl + return iface.Ssl + ? TcpTransport.CreateServer(in tcpConf, ssl: new HostAwareServerSslOptions(iface)) + : TcpTransport.CreateServer(in tcpConf, UseInlineScheduler); + + } + + + private void OnSocketConfiguring(Socket serverSock) + { + TcpConfigJson baseConf = _conf.Instance; + + serverSock.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.NoDelay, baseConf.NoDelay); + serverSock.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendBuffer, baseConf.TcpSendBufferSize); + serverSock.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, baseConf.TcpRecvBufferSize); + serverSock.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, ReuseAddress); + } + } +} diff --git a/apps/VNLib.WebServer/src/Transport/TcpTransport.cs b/apps/VNLib.WebServer/src/Transport/TcpTransport.cs new file mode 100644 index 0000000..fd6e7d1 --- /dev/null +++ b/apps/VNLib.WebServer/src/Transport/TcpTransport.cs @@ -0,0 +1,188 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: TcpTransport.cs +* +* TcpTransport.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Threading; +using System.IO.Pipelines; +using System.Net.Security; +using System.Threading.Tasks; +using System.Security.Authentication; + +using VNLib.Net.Http; +using VNLib.Net.Transport.Tcp; +using VNLib.Utils.Logging; + +namespace VNLib.WebServer.Transport +{ + /// <summary> + /// Creates the TCP/HTTP translation layer providers + /// </summary> + internal static class TcpTransport + { + /// <summary> + /// Creates a new <see cref="ITransportProvider"/> that will listen for tcp connections + /// </summary> + /// <param name="config">The server configuration</param> + /// <param name="inlineScheduler">Use the inline pipeline scheduler</param> + /// <returns>The configured <see cref="ITransportProvider"/></returns> + public static ITransportProvider CreateServer(ref readonly TCPConfig config, bool inlineScheduler) + { + //Create tcp server + TcpServer server = new (config, CreateCustomPipeOptions(in config, inlineScheduler)); + //Return provider + return new TcpTransportProvider(server); + } + + /// <summary> + /// Creates a new <see cref="ITransportProvider"/> that will listen for tcp connections + /// and use SSL + /// </summary> + /// <param name="config"></param> + /// <param name="ssl">The server authentication options</param> + /// <returns>The ssl configured transport context</returns> + public static ITransportProvider CreateServer(in TCPConfig config, SslServerAuthenticationOptions ssl) + { + /* + * SSL STREAM WORKAROUND + * + * The HttpServer impl calls Read() synchronously on the calling thread, + * it assumes that the call will make it synchronously to the underlying + * transport. SslStream calls ReadAsync() interally on the current + * synchronization context, which causes a deadlock... So the threadpool + * scheduler on the pipeline ensures that all continuations are run on the + * threadpool, which fixes this issue. + */ + + //Create tcp server + TcpServer server = new (config, CreateCustomPipeOptions(in config, false)); + //Return provider + return new SslTcpTransportProvider(server, ssl); + } + + private static PipeOptions CreateCustomPipeOptions(ref readonly TCPConfig config, bool inlineScheduler) + { + return new PipeOptions( + config.BufferPool, + //Noticable performance increase when using inline scheduler for reader (handles send operations) + readerScheduler: inlineScheduler ? PipeScheduler.Inline : PipeScheduler.ThreadPool, + writerScheduler: inlineScheduler ? PipeScheduler.Inline : PipeScheduler.ThreadPool, + pauseWriterThreshold: config.MaxRecvBufferData, + minimumSegmentSize: 8192, + useSynchronizationContext: false + ); + } + + /// <summary> + /// A TCP server transport provider class + /// </summary> + private class TcpTransportProvider(TcpServer Server) : ITransportProvider + { + protected ITcpListner? _listener; + protected CancellationTokenRegistration _reg; + + ///<inheritdoc/> + void ITransportProvider.Start(CancellationToken stopToken) + { + //TEMPORARY (unless it works) + if(_listener is not null) + { + throw new InvalidOperationException("The server has already been started."); + } + + //Start the server + _listener = Server.Listen(); + _reg = stopToken.Register(_listener.Close, false); + } + + ///<inheritdoc/> + public virtual async ValueTask<ITransportContext> AcceptAsync(CancellationToken cancellation) + { + //Wait for tcp event and wrap in ctx class + ITcpConnectionDescriptor descriptor = await _listener!.AcceptConnectionAsync(cancellation); + + return new TcpTransportContext(_listener, descriptor, descriptor.GetStream()); + } + + ///<inheritdoc/> + public override string ToString() => $"{Server.Config.LocalEndPoint} tcp/ip"; + } + + private sealed class SslTcpTransportProvider(TcpServer Server, SslServerAuthenticationOptions AuthOptions) + : TcpTransportProvider(Server) + { + /* + * An SslStream may throw a win32 exception with HRESULT 0x80090327 + * when processing a client certificate (I believe anyway) only + * an issue on some clients (browsers) + */ + + private const int UKNOWN_CERT_AUTH_HRESULT = unchecked((int)0x80090327); + + /// <summary> + /// An invlaid frame size may happen if data is recieved on an open socket + /// but does not contain valid SSL handshake data + /// </summary> + private const int INVALID_FRAME_HRESULT = unchecked((int)0x80131501); + + public override async ValueTask<ITransportContext> AcceptAsync(CancellationToken cancellation) + { + //Loop to handle ssl exceptions ourself + do + { + //Wait for tcp event and wrap in ctx class + ITcpConnectionDescriptor descriptor = await _listener.AcceptConnectionAsync(cancellation); + + //Create ssl stream and auth + SslStream stream = new(descriptor.GetStream(), false); + + try + { + //auth the new connection + await stream.AuthenticateAsServerAsync(AuthOptions, cancellation); + return new SslTcpTransportContext(_listener, descriptor, stream); + } + catch (AuthenticationException ae) when (ae.HResult == INVALID_FRAME_HRESULT) + { + Server.Config.Log.Debug("A TLS connection attempt was made but an invalid TLS frame was received"); + + await _listener.CloseConnectionAsync(descriptor, true); + await stream.DisposeAsync(); + + //continue listening loop + } + catch + { + await _listener.CloseConnectionAsync(descriptor, true); + await stream.DisposeAsync(); + throw; + } + } + while (true); + } + + ///<inheritdoc/> + public override string ToString() => $"{Server.Config.LocalEndPoint} tcp/ip (TLS enabled)"; + } + } +} diff --git a/apps/VNLib.WebServer/src/Transport/TcpTransportContext.cs b/apps/VNLib.WebServer/src/Transport/TcpTransportContext.cs new file mode 100644 index 0000000..2bb1b54 --- /dev/null +++ b/apps/VNLib.WebServer/src/Transport/TcpTransportContext.cs @@ -0,0 +1,93 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: TcpTransportContext.cs +* +* TcpTransportContext.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.IO; +using System.Net; +using System.Threading.Tasks; +using System.Runtime.CompilerServices; + +using VNLib.Net.Http; +using VNLib.Net.Transport.Tcp; + +namespace VNLib.WebServer.Transport +{ + /// <summary> + /// The TCP connection context + /// </summary> + internal class TcpTransportContext : ITransportContext + { + //Store static empty security info to pass in default case + private static readonly TransportSecurityInfo? EmptySecInfo; + + protected readonly ITcpConnectionDescriptor _descriptor; + + protected readonly Stream _connectionStream; + protected readonly IPEndPoint _localEndoint; + protected readonly IPEndPoint _remoteEndpoint; + protected readonly ITcpListner _server; + + public TcpTransportContext(ITcpListner server, ITcpConnectionDescriptor descriptor, Stream stream) + { + _descriptor = descriptor; + _connectionStream = stream; + _server = server; + //Get the endpoints + descriptor.GetEndpoints(out _localEndoint, out _remoteEndpoint); + } + + ///<inheritdoc/> + public virtual Stream ConnectionStream + { + [MethodImpl(MethodImplOptions.AggressiveInlining)] + get => _connectionStream; + } + + ///<inheritdoc/> + public virtual IPEndPoint LocalEndPoint + { + [MethodImpl(MethodImplOptions.AggressiveInlining)] + get => _localEndoint; + } + + ///<inheritdoc/> + public virtual IPEndPoint RemoteEndpoint + { + [MethodImpl(MethodImplOptions.AggressiveInlining)] + get => _remoteEndpoint; + } + + ///<inheritdoc/> + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public virtual async ValueTask CloseConnectionAsync() + { + //Close the stream before the descriptor + await _connectionStream.DisposeAsync(); + await _server.CloseConnectionAsync(_descriptor, true); + } + + //Ssl is not supported in this transport + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public virtual ref readonly TransportSecurityInfo? GetSecurityInfo() => ref EmptySecInfo; + } +} diff --git a/apps/VNLib.WebServer/src/VLogProvider.cs b/apps/VNLib.WebServer/src/VLogProvider.cs new file mode 100644 index 0000000..d20437f --- /dev/null +++ b/apps/VNLib.WebServer/src/VLogProvider.cs @@ -0,0 +1,84 @@ +/* +* Copyright (c) 2022 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: VLogProvider.cs +* +* VLogProvider.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.Linq; +using System.Runtime.CompilerServices; + +using Serilog; +using Serilog.Core; +using Serilog.Events; + +using VNLib.Utils; +using VNLib.Utils.Logging; + +namespace VNLib.WebServer +{ + internal sealed class VLogProvider : VnDisposeable, ILogProvider + { + private readonly Logger LogCore; + + public VLogProvider(LoggerConfiguration config) + { + LogCore = config.CreateLogger(); + } + public void Flush() { } + + public object GetLogProvider() => LogCore; + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public bool IsEnabled(LogLevel level) => LogCore.IsEnabled((LogEventLevel)level); + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public void Write(LogLevel level, string value) + { + LogCore.Write((LogEventLevel)level, value); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public void Write(LogLevel level, Exception exception, string value = "") + { + LogCore.Write((LogEventLevel)level, exception, value); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public void Write(LogLevel level, string value, params object[] args) + { + LogCore.Write((LogEventLevel)level, value, args); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public void Write(LogLevel level, string value, params ValueType[] args) + { + //Serilog logger supports passing valuetypes to avoid boxing objects + if (LogCore.IsEnabled((LogEventLevel)level)) + { + object[] ar = args.Select(a => (object)a).ToArray(); + LogCore.Write((LogEventLevel)level, value, ar); + } + } + + protected override void Free() => LogCore.Dispose(); + } +} diff --git a/apps/VNLib.WebServer/src/VNLib.WebServer.csproj b/apps/VNLib.WebServer/src/VNLib.WebServer.csproj new file mode 100644 index 0000000..37808b5 --- /dev/null +++ b/apps/VNLib.WebServer/src/VNLib.WebServer.csproj @@ -0,0 +1,88 @@ +<Project Sdk="Microsoft.NET.Sdk"> + + <PropertyGroup> + <OutputType>Exe</OutputType> + <TargetFramework>net8.0</TargetFramework> + <Nullable>enable</Nullable> + <AssemblyName>VNLib.WebServer</AssemblyName> + <RootNamespace>VNLib.WebServer</RootNamespace> + <ServerGarbageCollection>true</ServerGarbageCollection> + <ConcurrentGarbageCollection>true</ConcurrentGarbageCollection> + <RetainVMGarbageCollection>true</RetainVMGarbageCollection> + <StartupObject>VNLib.WebServer.Entry</StartupObject> + </PropertyGroup> + + <!-- Dotnet tool stuff --> + <PropertyGroup> + <PackAsTool>true</PackAsTool> + <ToolCommandName>VNLib.WebServer</ToolCommandName> + </PropertyGroup> + + <PropertyGroup> + <AnalysisLevel Condition="'$(BuildingInsideVisualStudio)' == true">latest-all</AnalysisLevel> + </PropertyGroup> + + <PropertyGroup> + <Authors>Vaughn Nugent</Authors> + <Company>Vaughn Nugent</Company> + <Description>A high performance, reference .NET 8 web/http server using VNLib.Core, with the VNLib.Plugins.Essentials web framework</Description> + <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> + <Product>VNLib.Webserver</Product> + <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/vnlib.core</PackageProjectUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/app/VNLib.Webserver/</RepositoryUrl> + <PackageReadmeFile>README.md</PackageReadmeFile> + <PackageLicenseFile>LICENSE</PackageLicenseFile> + </PropertyGroup> + + <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'"> + <Deterministic>False</Deterministic> + <DefineConstants>$(DefineConstants);USE_MCMASTER</DefineConstants> + </PropertyGroup> + + <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'"> + <Deterministic>False</Deterministic> + <DefineConstants>$(DefineConstants);USE_MCMASTER</DefineConstants> + </PropertyGroup> + + <ItemGroup> + <None Include="..\LICENSE"> + <Pack>True</Pack> + <PackagePath>\</PackagePath> + <CopyToOutputDirectory>Always</CopyToOutputDirectory> + </None> + <None Include="..\README.md"> + <Pack>True</Pack> + <PackagePath>\</PackagePath> + </None> + </ItemGroup> + + <ItemGroup> + <PackageReference Include="ErrorProne.NET.CoreAnalyzers" Version="0.1.2"> + <PrivateAssets>all</PrivateAssets> + <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> + </PackageReference> + <PackageReference Include="ErrorProne.NET.Structs" Version="0.1.2"> + <PrivateAssets>all</PrivateAssets> + <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> + </PackageReference> + <PackageReference Include="Serilog" Version="4.0.1" /> + <PackageReference Include="Serilog.Sinks.Console" Version="6.0.0" /> + <PackageReference Include="Serilog.Sinks.File" Version="6.0.0" /> + <PackageReference Include="YamlDotNet" Version="16.0.0" /> + </ItemGroup> + + <ItemGroup> + <ProjectReference Include="..\..\..\lib\Plugins.Essentials\src\VNLib.Plugins.Essentials.csproj" /> + <ProjectReference Include="..\..\..\lib\Net.Http\src\VNLib.Net.Http.csproj" /> + <ProjectReference Include="..\..\..\lib\Net.Transport.SimpleTCP\src\VNLib.Net.Transport.SimpleTCP.csproj" /> + <ProjectReference Include="..\..\..\lib\Plugins.Essentials.ServiceStack\src\VNLib.Plugins.Essentials.ServiceStack.csproj" /> + <ProjectReference Include="..\..\..\third-party\DotNetCorePlugins\src\McMaster.NETCore.Plugins.csproj" /> + </ItemGroup> + + <ItemGroup> + <None Update="sample.config.json"> + <CopyToOutputDirectory>Always</CopyToOutputDirectory> + </None> + </ItemGroup> + +</Project> diff --git a/apps/VNLib.WebServer/src/VirtualHosts/FileCache.cs b/apps/VNLib.WebServer/src/VirtualHosts/FileCache.cs new file mode 100644 index 0000000..de4efd9 --- /dev/null +++ b/apps/VNLib.WebServer/src/VirtualHosts/FileCache.cs @@ -0,0 +1,134 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: FileCache.cs +* +* FileCache.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Net; + +using VNLib.Net.Http; +using VNLib.Utils; +using VNLib.Utils.IO; + +namespace VNLib.WebServer +{ + /// <summary> + /// File the server will keep in memory and return to user when a specified error code is requested + /// </summary> + internal sealed class FileCache : VnDisposeable, IFSChangeHandler + { + private readonly string _filePath; + + public readonly HttpStatusCode Code; + + private Lazy<byte[]> _templateData; + + + /// <summary> + /// Catch an http error code and return the selected file to user + /// </summary> + /// <param name="code">Http status code to catch</param> + /// <param name="filePath">Path to file contating data to return to use on status code</param> + private FileCache(HttpStatusCode code, string filePath) + { + Code = code; + _filePath = filePath; + _templateData = new(LoadTemplateData); + } + + private byte[] LoadTemplateData() + { + //Get file data as binary + return File.ReadAllBytes(_filePath); + } + + /// <summary> + /// Gets a <see cref="IMemoryResponseReader"/> wrapper that may read a copy of the + /// file representation + /// </summary> + /// <returns>The <see cref="IMemoryResponseReader"/> wrapper around the file data</returns> + public IMemoryResponseReader GetReader() => new MemReader(_templateData.Value); + + + void IFSChangeHandler.OnFileChanged(FileSystemEventArgs e) + { + //Update lazy loader for new file update + _templateData = new(LoadTemplateData); + } + + protected override void Free() + { + //Unsubscribe from file watcher + FileWatcher.Unsubscribe(_filePath, this); + } + + /// <summary> + /// Create a new file cache for a specific error code + /// and begins listening for changes to the file + /// </summary> + /// <param name="code">The status code to produce the file for</param> + /// <param name="filePath">The path to the file to read</param> + /// <returns>The new <see cref="FileCache"/> instance if the file exists and is readable, null otherwise</returns> + public static FileCache? Create(HttpStatusCode code, string filePath) + { + //If the file does not exist, return null + if(!FileOperations.FileExists(filePath)) + { + return null; + } + + FileCache ff = new(code, filePath); + + //Subscribe to file changes + FileWatcher.Subscribe(filePath, ff); + + return ff; + } + + private sealed class MemReader : IMemoryResponseReader + { + private readonly byte[] _memory; + + private int _written; + + public int Remaining { get; private set; } + + internal MemReader(byte[] data) + { + //Store ref as memory + _memory = data; + Remaining = data.Length; + } + + public void Advance(int written) + { + _written += written; + Remaining -= written; + } + + void IMemoryResponseReader.Close() { } + + ReadOnlyMemory<byte> IMemoryResponseReader.GetMemory() => _memory.AsMemory(_written, Remaining); + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/VirtualHosts/IVirtualHostConfigBuilder.cs b/apps/VNLib.WebServer/src/VirtualHosts/IVirtualHostConfigBuilder.cs new file mode 100644 index 0000000..7e837e7 --- /dev/null +++ b/apps/VNLib.WebServer/src/VirtualHosts/IVirtualHostConfigBuilder.cs @@ -0,0 +1,35 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: IVirtualHostConfigBuilder.cs +* +* IVirtualHostConfigBuilder.cs is part of VNLib.WebServer which is +* part of the larger VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +namespace VNLib.WebServer +{ + internal interface IVirtualHostConfigBuilder + { + /// <summary> + /// Gets the base configuration for the virtualhost + /// </summary> + /// <returns></returns> + VirtualHostConfig GetBaseConfig(); + } +} diff --git a/apps/VNLib.WebServer/src/VirtualHosts/JsonWebConfigBuilder.cs b/apps/VNLib.WebServer/src/VirtualHosts/JsonWebConfigBuilder.cs new file mode 100644 index 0000000..7f3b488 --- /dev/null +++ b/apps/VNLib.WebServer/src/VirtualHosts/JsonWebConfigBuilder.cs @@ -0,0 +1,282 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: JsonWebConfigBuilder.cs +* +* JsonWebConfigBuilder.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Net; +using System.Data; +using System.Linq; +using System.Collections.Frozen; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using VNLib.Utils.Logging; +using VNLib.Utils.Extensions; + +using VNLib.WebServer.Config; +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer +{ + + internal sealed partial class JsonWebConfigBuilder(VirtualHostServerConfig VhConfig, ILogProvider logger) + : IVirtualHostConfigBuilder + { + //Use pre-compiled default regex + private static readonly Regex DefaultRootRegex = MyRegex(); + + ///<inheritdoc/> + public VirtualHostConfig GetBaseConfig() + { + //Declare the vh config + return new() + { + //File root is required + RootDir = new(VhConfig.DirPath!), + LogProvider = logger, + ExecutionTimeout = GetExecutionTimeout(VhConfig), + WhiteList = GetIpWhitelist(VhConfig), + DownStreamServers = GetDownStreamServers(VhConfig), + ExcludedExtensions = GetExlcudedExtensions(VhConfig), + DefaultFiles = GetDefaultFiles(VhConfig), + PathFilter = GetPathFilter(VhConfig), + CacheDefault = TimeSpan.FromSeconds(VhConfig.CacheDefaultTimeSeconds), + AdditionalHeaders = GetConfigHeaders(VhConfig), + SpecialHeaders = GetSpecialHeaders(VhConfig), + FailureFiles = GetFailureFiles(VhConfig), + FilePathCacheMaxAge = TimeSpan.MaxValue, + Hostnames = GetHostnames(VhConfig), + Transports = GetInterfaces(VhConfig), + BlackList = GetIpBlacklist(VhConfig), + }; + } + + private static string[] GetHostnames(VirtualHostServerConfig conf) + { + Validate.EnsureNotNull(conf.Hostnames, "Hostnames array was set to null, you must define at least one hostname"); + + foreach (string hostname in conf.Hostnames) + { + Validate.EnsureNotNull(hostname, "Hostname is null, all hostnames must be defined"); + } + + return conf.Hostnames; + } + + private static TransportInterface[] GetInterfaces(VirtualHostServerConfig conf) + { + Validate.EnsureNotNull(conf.Interfaces, "Interfaces array was set to null, you must define at least one network interface"); + Validate.Assert(conf.Interfaces.Length > 0, $"You must define at least one interface for host"); + + for(int i = 0; i < conf.Interfaces.Length; i++) + { + TransportInterface iFace = conf.Interfaces[i]; + + Validate.EnsureNotNull(iFace, $"Vrtual host interface [{i}] is undefined"); + + Validate.EnsureNotNull(iFace.Address, $"The interface IP address is required for interface [{i}]"); + Validate.EnsureValidIp(iFace.Address, $"The interface IP address is invalid for interface [{i}]"); + Validate.EnsureRange(iFace.Port, 1, 65535, "Interface port"); + } + + return conf.Interfaces; + } + + private static Regex GetPathFilter(VirtualHostServerConfig conf) + { + //Allow site to define a regex filter pattern + return conf.PathFilter is not null ? new(conf.PathFilter!) : DefaultRootRegex; + } + + private FrozenDictionary<HttpStatusCode, FileCache> GetFailureFiles(VirtualHostServerConfig conf) + { + //if a failure file array is specified, capure all files and + if (conf.ErrorFiles is null || conf.ErrorFiles.Length < 1) + { + return new Dictionary<HttpStatusCode, FileCache>().ToFrozenDictionary(); + } + + //Get the error files + IEnumerable<KeyValuePair<HttpStatusCode, string>> ffs = conf.ErrorFiles + .Select(static f => new KeyValuePair<HttpStatusCode, string>((HttpStatusCode)f.Code, f.Path!)); + + //Create the file cache dictionary + (HttpStatusCode, string, FileCache?)[] loadCache = ffs.Select(static kv => + { + FileCache? cached = FileCache.Create(kv.Key, kv.Value); + return (kv.Key, kv.Value, cached); + + }).ToArray(); + + //Only include files that exist and were loaded + int loadedFiles = loadCache + .Where(static loadCache => loadCache.Item3 != null) + .Count(); + + string[] notFoundFiles = loadCache + .Where(static loadCache => loadCache.Item3 == null) + .Select(static l => Path.GetFileName(l.Item2)) + .ToArray(); + + if (notFoundFiles.Length > 0) + { + logger.Warn("Failed to load error files {files} for host {hosts}", notFoundFiles, conf.Hostnames); + } + + //init frozen dictionary from valid cached files + return loadCache + .Where(static kv => kv.Item3 != null) + .ToDictionary(static kv => kv.Item1, static kv => kv.Item3!) + .ToFrozenDictionary(); + } + + private static FrozenSet<IPAddress> GetDownStreamServers(VirtualHostServerConfig conf) + { + //Find downstream servers + HashSet<IPAddress>? downstreamServers = null; + + //See if element is set + if (conf.DownstreamServers is not null) + { + //hash addresses, make is distinct + downstreamServers = conf.DownstreamServers + .Where(static addr => !string.IsNullOrWhiteSpace(addr)) + .Select(static addr => IPAddress.Parse(addr)) + .Distinct() + .ToHashSet(); + } + + return (downstreamServers ?? []).ToFrozenSet(); + } + + private static TimeSpan GetExecutionTimeout(VirtualHostServerConfig conf) + { + //Get the execution timeout + return TimeSpan.FromMilliseconds(conf.MaxExecutionTimeMs); + } + + private static FrozenSet<IPAddress>? GetIpWhitelist(VirtualHostServerConfig conf) + { + if(conf.Whitelist is null) + { + return null; + } + + //See if whitelist is defined, if so, get a distinct list of addresses + return conf.Whitelist + .Where(static addr => !string.IsNullOrWhiteSpace(addr)) + .Select(static addr => IPAddress.Parse(addr)) + .Distinct() + .ToHashSet() + .ToFrozenSet(); + } + + private static FrozenSet<IPAddress>? GetIpBlacklist(VirtualHostServerConfig conf) + { + if (conf.Blacklist is null) + { + return null; + } + + //See if whitelist is defined, if so, get a distinct list of addresses + return conf.Blacklist + .Where(static addr => !string.IsNullOrWhiteSpace(addr)) + .Select(static addr => IPAddress.Parse(addr)) + .Distinct() + .ToHashSet() + .ToFrozenSet(); + } + + private static FrozenSet<string> GetExlcudedExtensions(VirtualHostServerConfig conf) + { + //Get exlucded/denied extensions from config, ignore null strings + if (conf.DenyExtensions is not null) + { + return conf.DenyExtensions + .Where(static s => !string.IsNullOrWhiteSpace(s)) + .Distinct() + .ToHashSet() + .ToFrozenSet(StringComparer.OrdinalIgnoreCase); + } + else + { + return new HashSet<string>().ToFrozenSet(); + } + } + + private static IReadOnlyCollection<string> GetDefaultFiles(VirtualHostServerConfig conf) + { + if(conf.DefaultFiles is null) + { + return Array.Empty<string>(); + } + + //Get blocked extensions for the root + return conf.DefaultFiles + .Where(static s => !string.IsNullOrWhiteSpace(s)) + .Distinct() + .ToList(); + } + + private static KeyValuePair<string, string>[] GetConfigHeaders(VirtualHostServerConfig conf) + { + if (conf.Headers is null) + { + return []; + } + + //Enumerate kv headers + return conf.Headers + //Ignore empty keys or values + .Where(static p => !string.IsNullOrWhiteSpace(p.Key) && string.IsNullOrWhiteSpace(p.Value)) + //Exclude special headers + .Where(static p => !SpecialHeaders.SpecialHeader.Contains(p.Key, StringComparer.OrdinalIgnoreCase)) + .Select(static p => new KeyValuePair<string, string>(p.Key!, p.Value)) + .ToArray(); + } + + private static FrozenDictionary<string, string> GetSpecialHeaders(VirtualHostServerConfig conf) + { + //get the headers array property + if (conf.Headers is null) + { + return new Dictionary<string, string>().ToFrozenDictionary(); + } + + //Enumerate kv header + return conf.Headers + //Ignore empty keys or values + .Where(static p => !string.IsNullOrWhiteSpace(p.Key) && !string.IsNullOrWhiteSpace(p.Value)) + //Only include special headers + .Where(static p => SpecialHeaders.SpecialHeader.Contains(p.Key, StringComparer.OrdinalIgnoreCase)) + //Create the special dictionary + .ToDictionary(static k => k.Key, static k => k.Value, StringComparer.OrdinalIgnoreCase) + .ToFrozenDictionary(StringComparer.OrdinalIgnoreCase); + } + + + [GeneratedRegex(@"(\/\.\.)|(\\\.\.)|[\[\]^*<>|`~'\n\r\t\n]|(\s$)|^(\s)", RegexOptions.Compiled)] + private static partial Regex MyRegex(); + } +} diff --git a/apps/VNLib.WebServer/src/VirtualHosts/SpecialHeaders.cs b/apps/VNLib.WebServer/src/VirtualHosts/SpecialHeaders.cs new file mode 100644 index 0000000..a4a797d --- /dev/null +++ b/apps/VNLib.WebServer/src/VirtualHosts/SpecialHeaders.cs @@ -0,0 +1,71 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: SpecialHeaders.cs +* +* SpecialHeaders.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System.Runtime.CompilerServices; + +using VNLib.Net.Http; + +namespace VNLib.WebServer +{ + /// <summary> + /// Contains constants for internal/special headers by their name + /// </summary> + internal static class SpecialHeaders + { + public const string ContentSecPolicy = "Content-Security-Policy"; + public const string XssProtection = "X-XSS-Protection"; + public const string XContentOption = "X-Content-Type-Options"; + public const string Hsts = "Strict-Transport-Security"; + public const string Server = "Server"; + + /// <summary> + /// An array of the special headers to quickly compare against + /// </summary> + public static string[] SpecialHeader = + { + ContentSecPolicy, + XssProtection, + XContentOption, + Hsts, + Server + }; + + /// <summary> + /// Appends the special header by the given name, if it is present + /// in the current configruation's special headers + /// </summary> + /// <param name="config"></param> + /// <param name="server">The connection to set the response headers on</param> + /// <param name="headerName">The name of the special header to get</param> + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void TrySetSpecialHeader(this VirtualHostConfig config, IConnectionInfo server, string headerName) + { + //Try to get the special header value, + if(config.SpecialHeaders.TryGetValue(headerName, out string? headerValue)) + { + server.Headers.Append(headerName, headerValue); + } + } + } +} diff --git a/apps/VNLib.WebServer/src/VirtualHosts/VirtualHostConfig.cs b/apps/VNLib.WebServer/src/VirtualHosts/VirtualHostConfig.cs new file mode 100644 index 0000000..8af58aa --- /dev/null +++ b/apps/VNLib.WebServer/src/VirtualHosts/VirtualHostConfig.cs @@ -0,0 +1,101 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: VirtualHostConfig.cs +* +* VirtualHostConfig.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Net; +using System.Collections.Frozen; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.ServiceStack.Construction; + +using VNLib.WebServer.Config.Model; + +namespace VNLib.WebServer +{ + /// <summary> + /// Implementation of <see cref="IEpProcessingOptions"/> + /// with <see cref="VirtualHostHooks"/> extra processing options + /// </summary> + internal sealed class VirtualHostConfig : VirtualHostConfiguration, IEpProcessingOptions + { + public VirtualHostConfig() + { + //Update file attributes + AllowedAttributes = FileAttributes.Archive | FileAttributes.Compressed | FileAttributes.Normal | FileAttributes.ReadOnly; + DissallowedAttributes = FileAttributes.Device + | FileAttributes.Directory + | FileAttributes.Encrypted + | FileAttributes.Hidden + | FileAttributes.IntegrityStream + | FileAttributes.Offline + | FileAttributes.ReparsePoint + | FileAttributes.System; + } + + /// <summary> + /// A regex filter instance to filter incoming filesystem paths + /// </summary> + public Regex? PathFilter { get; init; } + + /// <summary> + /// The default response entity cache value + /// </summary> + public required TimeSpan CacheDefault { get; init; } + + /// <summary> + /// A collection of in-memory files to send in response to processing error + /// codes. + /// </summary> + public FrozenDictionary<HttpStatusCode, FileCache> FailureFiles { get; init; } = new Dictionary<HttpStatusCode, FileCache>().ToFrozenDictionary(); + + /// <summary> + /// Allows config to specify contant additional headers + /// </summary> + public KeyValuePair<string, string>[] AdditionalHeaders { get; init; } = Array.Empty<KeyValuePair<string, string>>(); + + /// <summary> + /// Contains internal headers used for specific purposes, cherrypicked from the config headers + /// </summary> + public FrozenDictionary<string, string> SpecialHeaders { get; init; } = new Dictionary<string, string>().ToFrozenDictionary(); + + /// <summary> + /// The array of interfaces the host wishes to listen on + /// </summary> + internal required TransportInterface[] Transports { get; init; } + + /// <summary> + /// An optional whitelist set of ipaddresses that are allowed to make connections to this site + /// </summary> + internal required FrozenSet<IPAddress>? WhiteList { get; init; } + + /// <summary> + /// An optional blacklist set of ipaddresses that are not allowed to make connections to this site + /// </summary> + internal required FrozenSet<IPAddress>? BlackList { get; init; } + + } +} diff --git a/apps/VNLib.WebServer/src/VirtualHosts/VirtualHostHooks.cs b/apps/VNLib.WebServer/src/VirtualHosts/VirtualHostHooks.cs new file mode 100644 index 0000000..e61b0a0 --- /dev/null +++ b/apps/VNLib.WebServer/src/VirtualHosts/VirtualHostHooks.cs @@ -0,0 +1,231 @@ +/* +* Copyright (c) 2024 Vaughn Nugent +* +* Library: VNLib +* Package: VNLib.WebServer +* File: VirtualHostHooks.cs +* +* VirtualHostHooks.cs is part of VNLib.WebServer which is part of the larger +* VNLib collection of libraries and utilities. +* +* VNLib.WebServer is free software: you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published +* by the Free Software Foundation, either version 2 of the License, +* or (at your option) any later version. +* +* VNLib.WebServer is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with VNLib.WebServer. If not, see http://www.gnu.org/licenses/. +*/ + +using System; +using System.IO; +using System.Net; +using System.Globalization; +using System.Runtime.CompilerServices; + +using VNLib.Net.Http; +using VNLib.Utils.Memory; +using VNLib.Utils.Extensions; +using VNLib.Plugins.Essentials; +using VNLib.Plugins.Essentials.Extensions; +using VNLib.Plugins.Essentials.ServiceStack.Construction; + +namespace VNLib.WebServer +{ + + internal sealed class VirtualHostHooks(VirtualHostConfig config) : IVirtualHostHooks + { + private const int FILE_PATH_BUILDER_BUFFER_SIZE = 4096; + + private static readonly string CultreInfo = CultureInfo.InstalledUICulture.Name; + + private readonly string DefaultCacheString = HttpHelpers.GetCacheString(CacheType.Public, (int)config.CacheDefault.TotalSeconds); + + public bool ErrorHandler(HttpStatusCode errorCode, IHttpEvent ev) + { + //Make sure the connection accepts html + if (ev.Server.Accepts(ContentType.Html) && config.FailureFiles.TryGetValue(errorCode, out FileCache? ff)) + { + ev.Server.SetNoCache(); + ev.CloseResponse(errorCode, ContentType.Html, ff.GetReader()); + return true; + } + return false; + } + + [MethodImpl(MethodImplOptions.AggressiveOptimization)] + public string TranslateResourcePath(string requestPath) + { + + /* + * This function must safely translate a request URL as "unsanitized" + * user input to a safe filesystem path for a desired resource. + * + * A user may supply a custom regex file as a first line of defense + * for illegal fs characters. + * + * It is safe to assume the path is a local path, (not absolute) so + * it should not contain an illegal FS scheme + */ + + requestPath = config.PathFilter?.Replace(requestPath, string.Empty) ?? requestPath; + + using UnsafeMemoryHandle<char> charBuffer = MemoryUtil.UnsafeAlloc<char>(FILE_PATH_BUILDER_BUFFER_SIZE); + + ForwardOnlyWriter<char> sb = new(charBuffer.Span); + + //Start with the root filename + sb.Append(config.RootDir.FullName); + + //Supply a "leading" dir separator character + if (requestPath[0] != '/') + { + sb.Append('/'); + } + + //Add the path (trimmed for whitespace) + sb.Append(requestPath); + + //Attmept to filter traversals + sb.Replace("..", string.Empty); + + //if were on windows, convert to windows directory separators + if (OperatingSystem.IsWindows()) + { + sb.Replace("/", "\\"); + } + else + { + sb.Replace("\\", "/"); + } + + /* + * DEFAULT: If no file extension is listed or, is not a / separator, then + * add a .html extension + */ + if (!Path.EndsInDirectorySeparator(requestPath) && !Path.HasExtension(requestPath)) + { + sb.AppendSmall(".html"); + } + + return sb.ToString(); + } + + public void PreProcessEntityAsync(HttpEntity entity, out FileProcessArgs args) + { + args = FileProcessArgs.Continue; + } + + public void PostProcessFile(HttpEntity entity, ref FileProcessArgs chosenRoutine) + { + //Do not respond to virtual processors + if (chosenRoutine == FileProcessArgs.VirtualSkip) + { + return; + } + + //Get-set the x-content options headers from the client config + config.TrySetSpecialHeader(entity.Server, SpecialHeaders.XContentOption); + + //Get the re-written url or + ReadOnlySpan<char> ext; + switch (chosenRoutine.Routine) + { + case FpRoutine.Deny: + case FpRoutine.Error: + case FpRoutine.NotFound: + case FpRoutine.Redirect: + { + ReadOnlySpan<char> filePath = entity.Server.Path.AsSpan(); + + //disable cache + entity.Server.SetNoCache(); + + //If the file is an html file or does not include an extension (inferred html) + ext = Path.GetExtension(filePath); + } + break; + case FpRoutine.ServeOther: + case FpRoutine.ServeOtherFQ: + { + ReadOnlySpan<char> filePath = chosenRoutine.Alternate.AsSpan(); + + //Use the alternate file path for extension + ext = Path.GetExtension(filePath); + + //Set default cache + ContentType ct = HttpHelpers.GetContentTypeFromFile(filePath); + SetCache(entity, ct); + } + break; + default: + { + ReadOnlySpan<char> filePath = entity.Server.Path.AsSpan(); + + //If the file is an html file or does not include an extension (inferred html) + ext = Path.GetExtension(filePath); + if (ext.IsEmpty) + { + //If no extension, use .html extension + SetCache(entity, ContentType.Html); + } + else + { + //Set default cache + ContentType ct = HttpHelpers.GetContentTypeFromFile(filePath); + SetCache(entity, ct); + } + } + break; + } + + //if the file is an html file, we are setting the csp and xss special headers + if (ext.IsEmpty || ext.Equals(".html", StringComparison.OrdinalIgnoreCase)) + { + //Get/set xss protection header + config.TrySetSpecialHeader(entity.Server, SpecialHeaders.XssProtection); + config.TrySetSpecialHeader(entity.Server, SpecialHeaders.ContentSecPolicy); + } + + //Set language of the server's os if the user code did not set it + if (!entity.Server.Headers.HeaderSet(HttpResponseHeader.ContentLanguage)) + { + entity.Server.Headers[HttpResponseHeader.ContentLanguage] = CultreInfo; + } + } + + private void SetCache(HttpEntity entity, ContentType ct) + { + //If request issued no cache request, set nocache headers + if (!entity.Server.NoCache()) + { + //Otherwise set caching based on the file extension type + switch (ct) + { + case ContentType.Css: + case ContentType.Jpeg: + case ContentType.Javascript: + case ContentType.Svg: + case ContentType.Img: + case ContentType.Png: + case ContentType.Apng: + case ContentType.Avi: + case ContentType.Avif: + case ContentType.Gif: + entity.Server.Headers[HttpResponseHeader.CacheControl] = DefaultCacheString; + return; + case ContentType.NonSupported: + return; + default: + break; + } + } + entity.Server.SetNoCache(); + } + } +}
\ No newline at end of file diff --git a/apps/VNLib.WebServer/src/sample.config.json b/apps/VNLib.WebServer/src/sample.config.json new file mode 100644 index 0000000..10ad783 --- /dev/null +++ b/apps/VNLib.WebServer/src/sample.config.json @@ -0,0 +1,171 @@ +{ + + //Host application config, config is loaded as a read-only DOM that is available + //to the host and loaded child plugins, all elements are available to plugins via the 'HostConfig' property + + "tcp": { + "keepalive_sec": 0, //How long to wait for a keepalive response before closing the connection (0 to disable tcp keepalive) + "keepalive_interval_sec": 0, //How long to wait between keepalive probes + "max_recv_size": 655360, //640k absolute maximum recv buffer (defaults to OS socket buffer size) + "max_connections": 50000, //Per listener instance + "backlog": 1000, //OS socket backlog, + + "tx_buffer": 65536, //OS socket send buffer size + "rx_buffer": 65536 //OS socket recv buffer size + }, + + "http": { + "default_version": "HTTP/1.1", //The defaut HTTP version to being requests with (does not support http/2 yet) + "multipart_max_buf_size": 20480, //The size of the buffer to use when parsing multipart/form data uploads + "multipart_max_size": 80240, //The maxium ammount of data (in bytes) allows for mulitpart/form data file uploads + "max_entity_size": 1024000, //Absolute maximum size (in bytes) of the request entity body (exludes headers) + "header_buf_size": 8128, //The buffer size to use when parsing headers (also the maxium request header size allowed) + "max_request_header_count": 50, //The maxium number of headers allowed in an HTTP request message + "max_connections": 5000, //The maxium number of allowed network connections, before 503s will be issued automatically and connections closed + "response_header_buf_size": 16384, //The size (in bytes) of the buffer used to store all response header data + "max_uploads_per_request": 10, //Max number of multi-part file uploads allowed per request + "keepalive_ms": 1000000, //Keepalive ms for HTTP1.1 keepalive connections + "recv_timeout_ms": 5000, //time (in ms) to wait for a response from an active connection in recv mode, before dropping it + "send_timeout_ms": 60000, //Time in ms to wait for the client to accept transport data before terminating the connection + + "compression": { + "enabled": true, //controls compression globally + "assembly": "", //A custom assembly path (ex: 'VNLib.Net.Compression.dll') + "max_size": 512000, //Maxium size of a response to compress before it's bypassed + "min_size": 2048 //Minium size of a response to compress, if smaller compression is bypassed + } + }, + + //Collection of objects to define hosts+interfaces to build server listeners from + "virtual_hosts": [ + { + + //The directory path for files served by this endpoint + "path": "path/to/website/root", + + //The hostname to listen for, "*" as wildcard, and "[system]" as the default hostname for the current machine. Must be unique + "hostnames": [ "*", "localhost" ], + + "trace": false, //Enables connection trace logging for this endpoint + "force_port_check": false, //If set, requires the port in the host header to match the transport port + + //Enable synthetic benchmarking + "benchmark": { + "enabled": false, + "random": true, + "size": 128 + }, + + //The interface to bind to, you may not mix TLS and non-TLS connections on the same interface + "interfaces": [ + { + "address": "0.0.0.0", + "port": 8080, + + "ssl": true, //Enables TLS for this interface for this host specifically + "certificate": "/path/to/cert.pfx|pem", //Cert may be pem or pfx (include private key in pfx, or include private key in a pem file) + "private_key": "/path/to/private_key.pem", //A pem encoded private key, REQUIRED if using a PEM certificate, may be encrypted with a password + "password": "plain-text-password", //An optional password for the ssl private key + "client_cert_required": false, //requires that any client connecting to this host present a valid certificate + "use_os_ciphers": false //Use the OS's ciphers instead of the hard-coded ciphers + } + ], + + + //Collection of "trusted" servers to allow proxy header support from + "downstream_servers": [ "127.0.0.1" ], + + /* + Specify a list of ip addresses that are allowed to connect to the server, 403 will be returned if connections are not on this list + whitelist works behind a trusted downstream server that supports X-Forwared-For headers + */ + "whitelist": [ "127.0.0.1" ], + + "blacklist": [ "127.0.0.1" ], //Individual IP addresses to blacklist + + //A list of file extensions to deny access to, if a resource is requested and has one of the following extensions, a 404 is returned + "deny_extensions": [ ".env", ".htaccess", ".php", ".gitignore" ], + + //The default file extensions to append to a resource that does not have a file extension + "default_files": [ "index.html", "index.htm" ], + + //Key-value headers object, some headers are special and are controlled by the vh processor + "headers": { + "header1": "header-value" + }, + + "cors": { + "enabled": true, //Enables cors protections for this host + "deny_cors_connections": false, //If true, all cors connections will be denied + "allowed_origins": [ "localhost:8089" ] + }, + + //A list of error file objects, files are loaded into memory (and watched for changes) and returned when the specified error code occurs + "error_files": [ + { + "code": 404, + "path": "path/to/404" + }, + { + "code": 403, + "path": "path/to/403" + } + ], + + //Default http cache time for files + "cache_default_sec": 864000, + + //Maxium ammount of time a request is allowed to be processed (includes loading or waiting for sessions) before operations will be cancelled and a 503 returned + "max_execution_time_ms": 20000 + } + ], + + + //Defines the directory where plugin's are to be loaded from + "plugins": { + "enabled": true, //Enable plugin loading + //Hot-reload creates collectable assemblies that allow full re-load support in the host application, should only be used for development purposes! + "hot_reload": false, + "reload_delay_sec": 2, + "paths": [ + "/path/to/plugins_dir" + ] + //"assets":"", + //"config_dir": "" + }, + + "logs": { + "sys_log": { + "enabled": false, + //"path": "path/to/syslog/file", + //"template": "serilog template for writing to file", + //"flush_sec": 5, + //"retained_files": 31, + //"file_size_limit": 10485760, + //"interval": "infinite" + }, + + "app_log": { + "enabled": false, + //"path": "path/to/applog/file", + //"template": "serilog template for writing to file", + //"flush_sec": 5, + //"retained_files": 31, + //"file_size_limit": 10485760, + //"interval": "infinite" + } + }, + + + //Global secrets object, used by the host and pluings for a specialized secrets + "secrets": { + + } + + //global or local configuration to define custom password hashing requirements instead of defaults + /* + "passwords": { + + } + */ +} diff --git a/lib/Hashing.Portable/src/VNLib.Hashing.Portable.csproj b/lib/Hashing.Portable/src/VNLib.Hashing.Portable.csproj index da61ae1..a4afc35 100644 --- a/lib/Hashing.Portable/src/VNLib.Hashing.Portable.csproj +++ b/lib/Hashing.Portable/src/VNLib.Hashing.Portable.csproj @@ -22,7 +22,7 @@ <Company>Vaughn Nugent</Company> <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Hashing.Portable</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Hashing.Portable</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Net.Compression/VNLib.Net.Compression/VNLib.Net.Compression.csproj b/lib/Net.Compression/VNLib.Net.Compression/VNLib.Net.Compression.csproj index c5b428e..2f49dda 100644 --- a/lib/Net.Compression/VNLib.Net.Compression/VNLib.Net.Compression.csproj +++ b/lib/Net.Compression/VNLib.Net.Compression/VNLib.Net.Compression.csproj @@ -24,7 +24,7 @@ <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <Description>.NET/8.0 dynamically loadable managed wrapper library for loading vnlib_compress native library. It provides an implementation of the IHttpCompressorManager interface for use with the VNLib.Net.Http library and servers wishing to support dynamic response compression.</Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Net.Compression</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Net.Compression</RepositoryUrl> <PackageReadmeFile>readme.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Net.Compression/vnlib_compress/CMakeLists.txt b/lib/Net.Compression/vnlib_compress/CMakeLists.txt index 6097a34..d0dbbb2 100644 --- a/lib/Net.Compression/vnlib_compress/CMakeLists.txt +++ b/lib/Net.Compression/vnlib_compress/CMakeLists.txt @@ -14,14 +14,13 @@ set(CMAKE_BUILD_TYPE "Release" CACHE STRING "The build configuration type") string(TOLOWER ${CMAKE_BUILD_TYPE} build_type) message(STATUS "Build type is '${build_type}'") -#export all header files to the main project -file(GLOB COMP_HEADERS *.h) +include(FetchContent) + +#the compression source file is required, all other sources will be added set(VNLIB_COMPRESS_SOURCES - compression.c + src/compression.c ) -include(FetchContent) - ############################### # # DOWNLOAD DEPENDENCIES @@ -50,7 +49,7 @@ if(ENABLE_BROTLI) include_directories(${lib_brotli_SOURCE_DIR}/c/include) #add the brotli source files to the project - list(APPEND VNLIB_COMPRESS_SOURCES feature_brotli.c) + list(APPEND VNLIB_COMPRESS_SOURCES src/feature_brotli.c) add_compile_definitions(VNLIB_COMPRESSOR_BROTLI_ENABLED) endif() @@ -77,7 +76,7 @@ if(ENABLE_ZLIB) include_directories(${lib_cf_zlib_SOURCE_DIR}) #enable the feature code for zlib and add the source files - list(APPEND VNLIB_COMPRESS_SOURCES feature_zlib.c) + list(APPEND VNLIB_COMPRESS_SOURCES src/feature_zlib.c) add_compile_definitions(VNLIB_COMPRESSOR_ZLIB_ENABLED) endif() @@ -104,12 +103,12 @@ endif() set(CMAKE_C_STANDARD_REQUIRED ON) if(COMPRESS_BUILD_SHARED) - add_library(${_COMP_PROJ_NAME} SHARED ${VNLIB_COMPRESS_SOURCES} ${COMP_HEADERS}) + add_library(${_COMP_PROJ_NAME} SHARED ${VNLIB_COMPRESS_SOURCES}) #enable position independent code (for shared libraries with exports) set_target_properties(${_COMP_PROJ_NAME} PROPERTIES POSITION_INDEPENDENT_CODE ON) else() set(_COMP_PROJ_NAME ${_COMP_PROJ_NAME}_static) #append static to the name - add_library(${_COMP_PROJ_NAME} STATIC ${VNLIB_COMPRESS_SOURCES} ${COMP_HEADERS}) + add_library(${_COMP_PROJ_NAME} STATIC ${VNLIB_COMPRESS_SOURCES}) endif() target_compile_features(${_COMP_PROJ_NAME} PRIVATE c_std_90) #force compiler to use c90 standard for library diff --git a/lib/Net.Compression/vnlib_compress/Taskfile.yaml b/lib/Net.Compression/vnlib_compress/Taskfile.yaml index f1d594e..2133c0b 100644 --- a/lib/Net.Compression/vnlib_compress/Taskfile.yaml +++ b/lib/Net.Compression/vnlib_compress/Taskfile.yaml @@ -15,9 +15,9 @@ tasks: default: cmds: - - cmd: echo "Building {{.PROJECT_NAME}}" + - cmd: echo "Building {{ .PROJECT_NAME }}" silent: true - - cmake -Bbuild/ -DCMAKE_BUILD_TYPE=Release {{.CLI_ARGS}} + - cmake -Bbuild/ -DCMAKE_BUILD_TYPE=Release {{ .CLI_ARGS }} - cmake --build build/ --config Release #called by ci pipline to build the winx64 project @@ -31,40 +31,88 @@ tasks: - cmake -B./build -DCI_PRECOMPILE=ON -DENABLE_RPMALLOC=ON #build for platform - - cmake --build build/ --config Debug - - cmake --build build/ --config Release + - cmake --build build/ --config debug + - cmake --build build/ --config release #when build succeeds, archive the output into a tgz postbuild_success: + cmds: + - cmd: powershell mkdir -Force 'bin/' + - task: pack_parallel + + pack_parallel: + internal: true vars: #required files to include in tar - TAR_FILES: "{{.PROJECT_NAME}}.dll {{.PROJECT_NAME}}.pdb {{.PROJECT_NAME}}.lib license.txt" + REL_TAR_FILES: + "{{ .PROJECT_NAME }}.dll + {{ .PROJECT_NAME }}.lib + compression.h + license.txt" + DEBUG_TAR_FILES: + "{{ .PROJECT_NAME }}.dll + {{ .PROJECT_NAME }}.pdb + {{ .PROJECT_NAME }}.lib + compression.h + license.txt" + + deps: + - task: pack_source + + - task: pack_artifacts + vars: { BUILD_MODE: 'debug', TAR_FILES: '{{ .DEBUG_TAR_FILES }}' } + - task: pack_artifacts + vars: { BUILD_MODE: 'release', TAR_FILES: '{{ .REL_TAR_FILES }}' } + + #packs up the binary artifacts for distribution + pack_artifacts: + internal: true + deps: + - task: embed + vars: { TARGET: './build/{{ .BUILD_MODE }}' } cmds: - - cmd: powershell mkdir -Force './bin' - #copy source code to target - - cmd: powershell -Command "tar --exclude build/* --exclude .vs/* --exclude bin/* --exclude third-party/* -czf bin/src.tgz ." - - #copy license file to debug and release output - - cmd: powershell -Command "cp ../LICENSE build/Debug/license.txt" - - cmd: powershell -Command "cp ../LICENSE build/Release/license.txt" + - cmd: cd build/{{ .BUILD_MODE }} && tar -czf '../../bin/msvc-x64-{{ .BUILD_MODE }}-{{ .PROJECT_NAME }}.tgz' {{ .TAR_FILES }} + + #add embeded resources to the binary output for distribution + embed: + internal: true + cmds: + - powershell cp ../LICENSE '{{ .TARGET }}/license.txt' + - powershell cp src/compression.h '{{ .TARGET }}/compression.h' + + #packages source code for distribution + pack_source: + internal: true + vars: + EXCLUDES: + + INCLUDES: + "src/* + LICENSE + CMakeLists.txt + Taskfile.yaml" + + cmds: + - powershell cp ../LICENSE 'LICENSE' -Force + #just pack up current directory, excluding build, bin, and git directories + - tar {{ .EXCLUDES }} -czf 'bin/src.tgz' {{ .INCLUDES }} + + - cmd: powershell rm LICENSE - #create static-build archives - - cd build/Debug && tar -czf ../../bin/msvc-x64-debug-{{.PROJECT_NAME}}.tgz {{.TAR_FILES}} - - cd build/Release && tar -czf ../../bin/msvc-x64-release-{{.PROJECT_NAME}}.tgz {{.TAR_FILES}} -#Remove the output dirs on clean + #Remove the output dirs on clean clean: ignore_error: true cmds: - for: [ bin/, build/ ] - cmd: powershell Remove-Item -Recurse '{{.ITEM}}' -Force + cmd: powershell rm -Recurse '{{ .ITEM }}' -Force clean-third-party: internal: false ignore_error: true cmds: - cmd: powershell rm -Recurse -Force 'build/_deps/' - platforms: [windows] + platforms: [ windows ] - cmd: rm -rf 'build/_deps/' - platforms: [linux, darwin] + platforms: [ linux, darwin ]
\ No newline at end of file diff --git a/lib/Net.Compression/vnlib_compress/package.json b/lib/Net.Compression/vnlib_compress/package.json index 6373511..bd6a815 100644 --- a/lib/Net.Compression/vnlib_compress/package.json +++ b/lib/Net.Compression/vnlib_compress/package.json @@ -3,7 +3,7 @@ "version": "0.1.0", "author": "Vaughn Nugent", "description": "A CMake cross platform native data compression library, provides brotli and zlib compressors in a single stream api", - "copyright": "Copyright \u00A9 2023 Vaughn Nugent", + "copyright": "Copyright \u00A9 2024 Vaughn Nugent", "company": "Vaughn Nugent", "repository": "https://github.com/VnUgE/VNLib.Core/tree/main/lib/Net.Compression/vnlib_compress", "output_dir": "bin" diff --git a/lib/Net.Compression/vnlib_compress/compression.c b/lib/Net.Compression/vnlib_compress/src/compression.c index 0cba998..0cba998 100644 --- a/lib/Net.Compression/vnlib_compress/compression.c +++ b/lib/Net.Compression/vnlib_compress/src/compression.c diff --git a/lib/Net.Compression/vnlib_compress/compression.h b/lib/Net.Compression/vnlib_compress/src/compression.h index 3d03145..3d03145 100644 --- a/lib/Net.Compression/vnlib_compress/compression.h +++ b/lib/Net.Compression/vnlib_compress/src/compression.h diff --git a/lib/Net.Compression/vnlib_compress/feature_brotli.c b/lib/Net.Compression/vnlib_compress/src/feature_brotli.c index 361c61a..361c61a 100644 --- a/lib/Net.Compression/vnlib_compress/feature_brotli.c +++ b/lib/Net.Compression/vnlib_compress/src/feature_brotli.c diff --git a/lib/Net.Compression/vnlib_compress/feature_brotli.h b/lib/Net.Compression/vnlib_compress/src/feature_brotli.h index 1f2090b..1f2090b 100644 --- a/lib/Net.Compression/vnlib_compress/feature_brotli.h +++ b/lib/Net.Compression/vnlib_compress/src/feature_brotli.h diff --git a/lib/Net.Compression/vnlib_compress/feature_zlib.c b/lib/Net.Compression/vnlib_compress/src/feature_zlib.c index a07f106..a07f106 100644 --- a/lib/Net.Compression/vnlib_compress/feature_zlib.c +++ b/lib/Net.Compression/vnlib_compress/src/feature_zlib.c diff --git a/lib/Net.Compression/vnlib_compress/feature_zlib.h b/lib/Net.Compression/vnlib_compress/src/feature_zlib.h index 2544d25..2544d25 100644 --- a/lib/Net.Compression/vnlib_compress/feature_zlib.h +++ b/lib/Net.Compression/vnlib_compress/src/feature_zlib.h diff --git a/lib/Net.Compression/vnlib_compress/util.h b/lib/Net.Compression/vnlib_compress/src/util.h index 292e3bf..292e3bf 100644 --- a/lib/Net.Compression/vnlib_compress/util.h +++ b/lib/Net.Compression/vnlib_compress/src/util.h diff --git a/lib/Net.Compression/vnlib_compress/vnlib_compress.vcxitems b/lib/Net.Compression/vnlib_compress/src/vnlib_compress.vcxitems index 0bfdcbf..7f5eb0e 100644 --- a/lib/Net.Compression/vnlib_compress/vnlib_compress.vcxitems +++ b/lib/Net.Compression/vnlib_compress/src/vnlib_compress.vcxitems @@ -25,9 +25,10 @@ <ClInclude Include="$(MSBuildThisFileDirectory)feature_zlib.h" /> </ItemGroup> <ItemGroup> - <Text Include="$(MSBuildThisFileDirectory)CMakeLists.txt" /> + <None Include="$(MSBuildThisFileDirectory)..\package.json" /> + <None Include="$(MSBuildThisFileDirectory)..\Taskfile.yaml" /> </ItemGroup> <ItemGroup> - <None Include="$(MSBuildThisFileDirectory)Taskfile.yaml" /> + <Text Include="$(MSBuildThisFileDirectory)..\CMakeLists.txt" /> </ItemGroup> </Project>
\ No newline at end of file diff --git a/lib/Net.Http/src/Core/TransportManager.cs b/lib/Net.Http/src/Core/TransportManager.cs index 2632fc5..8f1f946 100644 --- a/lib/Net.Http/src/Core/TransportManager.cs +++ b/lib/Net.Http/src/Core/TransportManager.cs @@ -47,7 +47,7 @@ namespace VNLib.Net.Http.Core Debug.Assert(_stream != null, "Transport stream was accessed but was set to null"); return _stream; } - set => _stream = value; + private set => _stream = value; } public IBufferWriter<byte> Writer @@ -57,7 +57,7 @@ namespace VNLib.Net.Http.Core Debug.Assert(_asWriter != null, "Transport buffer writer accessed but the writer is null"); return _asWriter; } - set => _asWriter = value; + private set => _asWriter = value; } #else diff --git a/lib/Net.Http/src/VNLib.Net.Http.csproj b/lib/Net.Http/src/VNLib.Net.Http.csproj index e57f784..d9117f4 100644 --- a/lib/Net.Http/src/VNLib.Net.Http.csproj +++ b/lib/Net.Http/src/VNLib.Net.Http.csproj @@ -22,7 +22,7 @@ <Description>Provides a high performance HTTP 0.9-1.1 application processing layer for handling transport *agnostic connections and asynchronous event support for applications serving HTTP requests such as web content. This library has a large focus on low/no GC allocations using unmanaged memory support provided by the VNLib.Utils library. No external dependencies outside of the VNLib ecosystem are required. The VNLib.Plugins and VNLib.Plugins.Essentials libraries are highly recommended for serving web content.</Description> <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Net.Http</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Net.Http</RepositoryUrl> <PackageReadmeFile>readme.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Net.Messaging.FBM/src/VNLib.Net.Messaging.FBM.csproj b/lib/Net.Messaging.FBM/src/VNLib.Net.Messaging.FBM.csproj index 897bca5..8fc2fb0 100644 --- a/lib/Net.Messaging.FBM/src/VNLib.Net.Messaging.FBM.csproj +++ b/lib/Net.Messaging.FBM/src/VNLib.Net.Messaging.FBM.csproj @@ -20,7 +20,7 @@ <Description>Fixed Buffer Messaging protocol class library. Implements a high-performance asynchronous request/response messaging architecture build on top of HTTP websockets. Provides client and server data structures.</Description> <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Net.Messaging.FBM</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Net.Messaging.FBM</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Net.Rest.Client/src/VNLib.Net.Rest.Client.csproj b/lib/Net.Rest.Client/src/VNLib.Net.Rest.Client.csproj index 95f7e05..37ae341 100644 --- a/lib/Net.Rest.Client/src/VNLib.Net.Rest.Client.csproj +++ b/lib/Net.Rest.Client/src/VNLib.Net.Rest.Client.csproj @@ -20,7 +20,7 @@ <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <Description>Provides a RestClient connection pool using the Utils.ObjectRental framework for reusing RestClients with a common configuration to execute RestRequests against. This library also implements an OAuth2 authenticator for the RestSharp IAuthenticator using the client credentials method.</Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Net.Rest.Client</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Net.Rest.Client</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Net.Transport.SimpleTCP/src/VNLib.Net.Transport.SimpleTCP.csproj b/lib/Net.Transport.SimpleTCP/src/VNLib.Net.Transport.SimpleTCP.csproj index 630a0db..a2fc268 100644 --- a/lib/Net.Transport.SimpleTCP/src/VNLib.Net.Transport.SimpleTCP.csproj +++ b/lib/Net.Transport.SimpleTCP/src/VNLib.Net.Transport.SimpleTCP.csproj @@ -22,7 +22,7 @@ <Company>Vaughn Nugent</Company> <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Net.Transport.SimpleTCP</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Net.Transport.SimpleTCP</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Plugins.Essentials.ServiceStack/src/VNLib.Plugins.Essentials.ServiceStack.csproj b/lib/Plugins.Essentials.ServiceStack/src/VNLib.Plugins.Essentials.ServiceStack.csproj index 4aedb80..b5dc949 100644 --- a/lib/Plugins.Essentials.ServiceStack/src/VNLib.Plugins.Essentials.ServiceStack.csproj +++ b/lib/Plugins.Essentials.ServiceStack/src/VNLib.Plugins.Essentials.ServiceStack.csproj @@ -20,7 +20,7 @@ <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <Description>Provides data structures for building a dynamic HTTP servicing stack with runtime plugin loading, dynamic endpoint routing, middleware* configuration, and multi-server managment.</Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Plugins.Essentials.ServiceStack</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Plugins.Essentials.ServiceStack</RepositoryUrl> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Plugins.Essentials/src/VNLib.Plugins.Essentials.csproj b/lib/Plugins.Essentials/src/VNLib.Plugins.Essentials.csproj index 3a21ac9..45a4d50 100644 --- a/lib/Plugins.Essentials/src/VNLib.Plugins.Essentials.csproj +++ b/lib/Plugins.Essentials/src/VNLib.Plugins.Essentials.csproj @@ -22,7 +22,7 @@ <Description>Provides essential web, sessions, users abstractions for building extensable web applications with satefull sessions, user based intraction with login and account security extensions.</Description> <PackageTags>VNLib, Plugins, VNLib.Plugins.Essentials, Essentials, Essential Plugins, HTTP Essentials, OAuth2</PackageTags> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Plugins.Essentials</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Plugins.Essentials</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Plugins.PluginBase/src/VNLib.Plugins.PluginBase.csproj b/lib/Plugins.PluginBase/src/VNLib.Plugins.PluginBase.csproj index 2f4abfa..d1207c2 100644 --- a/lib/Plugins.PluginBase/src/VNLib.Plugins.PluginBase.csproj +++ b/lib/Plugins.PluginBase/src/VNLib.Plugins.PluginBase.csproj @@ -20,7 +20,7 @@ <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <Description>A base class for VNLib.Plugins that provides all supported loading functionalities, with built-in logging, loading, task-managment and endpoint routing.</Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Plugins.PluginBase</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Plugins.PluginBase</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Plugins.Runtime/src/VNLib.Plugins.Runtime.csproj b/lib/Plugins.Runtime/src/VNLib.Plugins.Runtime.csproj index 002716f..6d18bef 100644 --- a/lib/Plugins.Runtime/src/VNLib.Plugins.Runtime.csproj +++ b/lib/Plugins.Runtime/src/VNLib.Plugins.Runtime.csproj @@ -20,7 +20,7 @@ <Copyright>Copyright © 2024 Vaughn Nugent</Copyright> <Description>A runtime plugin loader for .NET. Allows runtime loading and tracking of .NET assemblies that export the VNLib.Plugin.IPlugin interface.</Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Plugins.Runtime</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Plugins.Runtime</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Plugins/src/VNLib.Plugins.csproj b/lib/Plugins/src/VNLib.Plugins.csproj index 343d86f..4ac6366 100644 --- a/lib/Plugins/src/VNLib.Plugins.csproj +++ b/lib/Plugins/src/VNLib.Plugins.csproj @@ -23,7 +23,7 @@ <Description>Provides a standard interface for building dynamically loadable plugins and asynchronus web endpoint processing, compatible with the VNLib.Plugins.Runtime loader library. </Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Plugins</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Plugins</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/lib/Utils.Cryptography/argon2/.gitattributes b/lib/Utils.Cryptography/argon2/.gitattributes deleted file mode 100644 index 177bed9..0000000 --- a/lib/Utils.Cryptography/argon2/.gitattributes +++ /dev/null @@ -1,10 +0,0 @@ -# Export ignore -.gitattributes export-ignore -.gitignore export-ignore -.travis.yml export-ignore -appveyor.yml export-ignore -export.sh export-ignore -latex/* export-ignore - -# Linguist documentation -latex/* linguist-documentation diff --git a/lib/Utils.Cryptography/argon2/.gitignore b/lib/Utils.Cryptography/argon2/.gitignore deleted file mode 100644 index 2bb1416..0000000 --- a/lib/Utils.Cryptography/argon2/.gitignore +++ /dev/null @@ -1,22 +0,0 @@ -argon2 -libargon2.a -libargon2.so* -libargon2.dylib -libargon2.pc -.DS_Store -src/*.o -src/blake2/*.o -genkat -.idea -*.pyc -testcase -*.gcda -*.gcno -*.gcov -bench -vs2015/build -Argon2.sdf -Argon2.VC.opendb -*.zip -*.tar.gz -tags diff --git a/lib/Utils.Cryptography/argon2/.travis.yml b/lib/Utils.Cryptography/argon2/.travis.yml deleted file mode 100644 index 265fc48..0000000 --- a/lib/Utils.Cryptography/argon2/.travis.yml +++ /dev/null @@ -1,25 +0,0 @@ -language: c - -compiler: - - clang - - gcc - -os: - - linux - - osx - -# Clang on Linux needs to run in a VM to use ASAN. -# See: https://github.com/travis-ci/travis-ci/issues/9033 -matrix: - exclude: - - compiler: clang - os: linux - include: - - compiler: clang - os: linux - sudo: true - -script: make && make testci - -after_success: - - bash <(curl -s https://codecov.io/bash) diff --git a/lib/Utils.Cryptography/argon2/Taskfile.yaml b/lib/Utils.Cryptography/argon2/Taskfile.yaml index 73fb8fa..d08d3b6 100644 --- a/lib/Utils.Cryptography/argon2/Taskfile.yaml +++ b/lib/Utils.Cryptography/argon2/Taskfile.yaml @@ -19,68 +19,84 @@ tasks: cmds: #build with defaults - task: build - - cmd: echo "Your Argon2 dll file can be found in '{{.USER_WORKING_DIR}}/build'" + - cmd: echo "Your Argon2 library file can be found in '{{ .USER_WORKING_DIR }}/build'" silent: true build: cmds: #init cmake build with greedy enabled - - cmake -B./build - - - cmake --build build/ --config Debug - - cmake --build build/ --config Release + - cmake -B./build + - cmake --build build/ --config debug + - cmake --build build/ --config release postbuild_success: - vars: - #required files to include in tar - TAR_FILES: "license.txt readme.txt argon2.h" - cmds: #make bin dir - - cmd: powershell -Command "New-Item -Type Directory -Force -Path './bin'" + - cmd: powershell mkdir -Force 'bin/' ignore_error: true - - #add embeded resources to output dirs - - task: embed - vars: - TARGET: './build/Debug' - - - task: embed - vars: - TARGET: './build/Release' - - - #static debug lib - - cd build/Debug && tar -czf '../../bin/msvc-x64-debug-{{.PROJECT_NAME}}-static.tgz' {{.PROJECT_NAME}}_static.lib {{.TAR_FILES}} {{.PROJECT_NAME}}_static.pdb - #dynamic debug lib - - cd build/Debug && tar -czf '../../bin/msvc-x64-debug-{{.PROJECT_NAME}}.tgz' {{.PROJECT_NAME}}.dll {{.TAR_FILES}} {{.PROJECT_NAME}}.pdb - #release static lib - - cd build/Release && tar -czf '../../bin/msvc-x64-release-{{.PROJECT_NAME}}-static.tgz' {{.PROJECT_NAME}}_static.lib {{.TAR_FILES}} - #release dll - - cd build/Release && tar -czf '../../bin/msvc-x64-release-{{.PROJECT_NAME}}.tgz' {{.PROJECT_NAME}}.dll {{.TAR_FILES}} + - task: parallel_postbuild + + parallel_postbuild: + internal: true + vars: + #required files to include in tar + BIN_FILES: + license.txt + readme.txt + argon2.h + '{{ .PROJECT_NAME }}.dll' + + deps: + - task: pack_source + + - task: pack_artifacts + vars: { BUILD_MODE: 'debug', TAR_FILES: '{{ .BIN_FILES }} {{ .PROJECT_NAME }}.pdb' } + - task: pack_artifacts + vars: { BUILD_MODE: 'release', TAR_FILES: '{{ .BIN_FILES }}' } - #source code - - task: pack_source + #packs up the binary artifacts for distribution + pack_artifacts: + internal: true + deps: + - task: embed + vars: { TARGET: './build/{{ .BUILD_MODE }}' } + cmds: + - cmd: cd build/{{ .BUILD_MODE }} && tar -czf '../../bin/msvc-x64-{{ .BUILD_MODE }}-{{ .PROJECT_NAME }}.tgz' {{ .TAR_FILES }} + #add embeded resources to the binary output for distribution embed: + internal: true cmds: - #add license file - - powershell -Command "Copy-Item -Path ./license -Destination '{{.TARGET}}/license.txt'" - #add readme file - - powershell -Command "Copy-Item -Path ./build.readme.txt -Destination '{{.TARGET}}/readme.txt'" - #add argon2 header file - - powershell -Command "Copy-Item -Path ./include/argon2.h -Destination '{{.TARGET}}/argon2.h'" + - powershell cp LICENSE '{{.TARGET}}/license.txt' + - powershell cp build.readme.txt '{{.TARGET}}/readme.txt' + - powershell cp include/argon2.h '{{.TARGET}}/argon2.h' + #packages source code for distribution pack_source: + internal: true + vars: + EXCLUDES: + + INCLUDES: + src/* + include/* + CHANGELOG.md + CMakeLists.txt + README.md + LICENSE + Argon2.sln + libargon2.pc.in + Makefile + Taskfile.yaml + cmds: #just pack up current directory, excluding build, bin, and git directories - - powershell -Command "tar --exclude build/* --exclude bin/* -czf bin/src.tgz ." - + - tar {{ .EXCLUDES }} -czf 'bin/src.tgz' {{ .INCLUDES }} clean: ignore_error: true cmds: - - cmd: powershell -Command "Remove-Item -Recurse './bin'" - - cmd: powershell -Command "Remove-Item -Recurse './build'"
\ No newline at end of file + - for: [ bin/, build/ ] + cmd: powershell -Command "Remove-Item -Recurse '{{ .ITEM }}'" diff --git a/lib/Utils.Cryptography/argon2/appveyor.yml b/lib/Utils.Cryptography/argon2/appveyor.yml deleted file mode 100644 index fb1f048..0000000 --- a/lib/Utils.Cryptography/argon2/appveyor.yml +++ /dev/null @@ -1,25 +0,0 @@ -os: Visual Studio 2015 - -environment: - matrix: - - platform: x86 - configuration: Debug - - platform: x86 - configuration: Release - - platform: x64 - configuration: Debug - - platform: x64 - configuration: Release - -matrix: - fast_finish: false - -build: - parallel: true - project: Argon2.sln - verbosity: minimal - -test_script: - - ps: kats\test.ps1 - - ps: if ("Release" -eq $env:configuration) { vs2015\build\Argon2OptTestCI.exe } - - ps: if ("Release" -eq $env:configuration) { vs2015\build\Argon2RefTestCI.exe } diff --git a/lib/Utils.Cryptography/argon2/argon2-specs.pdf b/lib/Utils.Cryptography/argon2/argon2-specs.pdf Binary files differdeleted file mode 100644 index d916af6..0000000 --- a/lib/Utils.Cryptography/argon2/argon2-specs.pdf +++ /dev/null diff --git a/lib/Utils.Cryptography/argon2/export.sh b/lib/Utils.Cryptography/argon2/export.sh deleted file mode 100644 index 9757b10..0000000 --- a/lib/Utils.Cryptography/argon2/export.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/sh - -FILE=`date "+%Y%m%d"` -BRANCH=master - -git archive --format zip --output $FILE.zip $BRANCH -git archive --format tar.gz --output $FILE.tar.gz $BRANCH diff --git a/lib/Utils.Memory/NativeHeapApi/src/NativeHeapApi.h b/lib/Utils.Memory/NativeHeapApi/src/NativeHeapApi.h index 1738d19..b54313a 100644 --- a/lib/Utils.Memory/NativeHeapApi/src/NativeHeapApi.h +++ b/lib/Utils.Memory/NativeHeapApi/src/NativeHeapApi.h @@ -51,7 +51,7 @@ #ifdef _P_IS_WINDOWS #define VNLIB_HEAP_API __declspec(dllimport) #else - #define VNLIB_HEAP_API + #define VNLIB_HEAP_API extern #endif /* _P_IS_WINDOWS */ #endif /* !VNLIB_EXPORTING */ #endif /* !VNLIB_EXPORT */ @@ -72,12 +72,9 @@ typedef enum HeapCreationFlags HEAP_CREATION_SUPPORTS_REALLOC = 0x08 } HeapCreationFlags; -#ifdef _P_IS_WINDOWS -typedef void* LPVOID; -#endif /* !WIN32 */ - /* The vnlib ERRNO type, integer/process dependent, -internally represented as a pointer */ + internally represented as a pointer +*/ typedef void* ERRNO; /* A pointer to a heap structure that was stored during heap creation */ diff --git a/lib/Utils.Memory/vnlib_mimalloc/NativeHeapApi.h b/lib/Utils.Memory/vnlib_mimalloc/NativeHeapApi.h index 1738d19..b54313a 100644 --- a/lib/Utils.Memory/vnlib_mimalloc/NativeHeapApi.h +++ b/lib/Utils.Memory/vnlib_mimalloc/NativeHeapApi.h @@ -51,7 +51,7 @@ #ifdef _P_IS_WINDOWS #define VNLIB_HEAP_API __declspec(dllimport) #else - #define VNLIB_HEAP_API + #define VNLIB_HEAP_API extern #endif /* _P_IS_WINDOWS */ #endif /* !VNLIB_EXPORTING */ #endif /* !VNLIB_EXPORT */ @@ -72,12 +72,9 @@ typedef enum HeapCreationFlags HEAP_CREATION_SUPPORTS_REALLOC = 0x08 } HeapCreationFlags; -#ifdef _P_IS_WINDOWS -typedef void* LPVOID; -#endif /* !WIN32 */ - /* The vnlib ERRNO type, integer/process dependent, -internally represented as a pointer */ + internally represented as a pointer +*/ typedef void* ERRNO; /* A pointer to a heap structure that was stored during heap creation */ diff --git a/lib/Utils.Memory/vnlib_mimalloc/Taskfile.yaml b/lib/Utils.Memory/vnlib_mimalloc/Taskfile.yaml index 2d816e3..c1ded6d 100644 --- a/lib/Utils.Memory/vnlib_mimalloc/Taskfile.yaml +++ b/lib/Utils.Memory/vnlib_mimalloc/Taskfile.yaml @@ -9,75 +9,108 @@ version: '3' vars: - PROJECT_NAME: 'vnlib_mimalloc' - MODULE_NAME: 'vnlib.core' + DEP_NAME: 'mimalloc' + PROJECT_NAME: 'vnlib_{{ .DEP_NAME }}' + ARTIFACTS_OUT: '{{ .PROJECT_DIR }}/{{ .BINARY_DIR }}' tasks: default: desc: "Builds the entire project from source code without using the VNBuild build system for target machines" cmds: - #init cmake build with greedy enabled - - cmake -Bbuild/ -DCMAKE_BUILD_TYPE=Release -DENABLE_GREEDY=1 {{.CLI_ARGS}} - - cmake --build build/ --config Release - - cmd: echo "Your vnlib_mimalloc library file can be found in '{{.USER_WORKING_DIR}}/build'" - silent: true - + #init cmake build with greedy enabled + - task: build-internal + vars: + BUILD_TYPE: "Release" + CLI_ARGS: "-DENABLE_GREEDY=1 {{ .CLI_ARGS }}" + + - cmd: echo "Your {{ .PROJECT_NAME }} library file can be found in '{{ .USER_WORKING_DIR }}/build'" + silent: true + + debug: + cmds: + - task: build-internal + vars: + BUILD_TYPE: "debug" + CLI_ARGS: "-DENABLE_GREEDY=1 {{ .CLI_ARGS }}" + build: - cmds: - #build our library - - cmake -B./build -DENABLE_GREEDY=1 {{.CLI_ARGS}} - - cmake --build build/ --config Debug - - cmake --build build/ --config Release + desc: 'DO NOT USE! This function is used internally during releases.' + cmds: + #init cmake build with greedy enabled + - cmake -B./build -DENABLE_GREEDY=1 {{ .CLI_ARGS }} + - cmake --build build/ --config debug + - cmake --build build/ --config release + + build-internal: + internal: true + cmds: + #build with defaults + #init cmake build with greedy enabled + - cmake -Bbuild/ -DCMAKE_BUILD_TYPE={{ .BUILD_TYPE }} {{ .CLI_ARGS }} + - cmake --build build/ --config {{ .BUILD_TYPE }} postbuild_success: - vars: - #required files to include in tar - TAR_FILES: "license.txt readme.txt mimalloc_license.txt" - cmds: #make bin dir - - cmd: powershell -Command "New-Item -Type Directory -Force -Path './bin'" + - cmd: powershell mkdir -Force '{{ .ARTIFACTS_OUT }}' ignore_error: true + + - task: pack_parallel - #get licenses for debug - - task: licenses - vars: - TARGET: './build/Debug' - - - task: licenses - vars: - TARGET: './build/Release' - - #dynamic debug lib - - cd build/Debug && tar -czf '../../bin/msvc-x64-debug-{{.PROJECT_NAME}}.tgz' {{.PROJECT_NAME}}.dll {{.TAR_FILES}} {{.PROJECT_NAME}}.pdb - - #release dll - - cd build/Release && tar -czf '../../bin/msvc-x64-release-{{.PROJECT_NAME}}.tgz' {{.PROJECT_NAME}}.dll {{.TAR_FILES}} - - #source code - - task: pack_source + pack_parallel: + internal: true + deps: + - task: pack_source + - task: pack_artifacts + vars: { BUILD_TYPE: 'debug' } + - task: pack_artifacts + vars: { BUILD_TYPE: 'release' } - licenses: + #packs binary artifacts for distribution + pack_artifacts: + internal: true + deps: + - task: pack_resources + vars: { TARGET: './build/{{ .BUILD_TYPE }}' } cmds: - #add mimalloc license to binary output - - powershell -Command "Copy-Item -Path vendor/license -Destination '{{.TARGET}}/mimalloc_license.txt'" - #add my license file - - powershell -Command "Copy-Item -Path ./license -Destination '{{.TARGET}}/license.txt'" - #add readme file - - powershell -Command "Copy-Item -Path ./build.readme.txt -Destination '{{.TARGET}}/readme.txt'" + - cmd: cd build/{{ .BUILD_TYPE }} && tar -czf '{{ .ARTIFACTS_OUT }}/msvc-x64-{{ .BUILD_TYPE }}-{{ .PROJECT_NAME }}.tgz' . + + pack_resources: + internal: true + cmds: + - powershell cp NativeHeapApi.h '{{ .TARGET }}/' + - powershell cp vendor/license '{{ .TARGET }}/{{ .DEP_NAME }}_license.txt' + - powershell cp license '{{ .TARGET }}/license.txt' + - powershell cp build.readme.txt '{{ .TARGET }}/readme.txt' pack_source: - dir: '{{.USER_WORKING_DIR}}' + internal: true + desc: "Packs up the project source code and creates a tarball in the builds binary directory" + vars: + TARGET_SOURCE: '{{ .PROJECT_DIR }}/{{ .BINARY_DIR }}/src.tgz' + SOURCE_FILES: + CMakeLists.txt + Taskfile.yaml + vendor + LICENSE + NativeHeapApi.h + '{{ .PROJECT_NAME }}.c' + cmds: - #pack mimalloc source code and create the archive - - powershell -Command "tar --exclude build/* --exclude bin/* --exclude vendor/test/ --exclude vendor/out/ -czf 'bin/src.tgz' ." + #tar up the source + - tar -czf '{{ .TARGET_SOURCE }}' {{ .SOURCE_FILES | join " " }} clean: ignore_error: true cmds: - for: [ bin/, build/ ] - cmd: powershell Remove-Item -Recurse '{{.ITEM}}' -Force - -
\ No newline at end of file + cmd: powershell rm -Recurse -Force '{{.ITEM}}' + + #called by module for initializing the project, it's not internal but hidden + dev-init: + internal: false + cmds: + - task: debug +
\ No newline at end of file diff --git a/lib/Utils.Memory/vnlib_mimalloc/build.readme.txt b/lib/Utils.Memory/vnlib_mimalloc/build.readme.txt index cd003b6..2edae23 100644 --- a/lib/Utils.Memory/vnlib_mimalloc/build.readme.txt +++ b/lib/Utils.Memory/vnlib_mimalloc/build.readme.txt @@ -25,9 +25,9 @@ PREREQUISITES: NOTE: If you have any mimalloc specific CMake options you want to use, when running task, you can pass them as -an the MIMALLOC_CMAKE_ARGS env variable. +on the command line following a double hyphen '--' -Example: >task MIMALLOC_CMAKE_ARGS="-DMI_SECURE=ON" (enable secure mode for mimalloc) +Example: >task -- -DMI_SECURE=ON (enable secure mode for mimalloc) See: https://microsoft.github.io/mimalloc/build.html for more information on mimalloc specific CMake options. @@ -36,6 +36,12 @@ BUILDING: 2. Navigate to directory containing the Taskfile.yaml file in the root 3. Run the default task: > task (yes literally just type "task" and hit enter if you installed Task gobally) +You may build a debug version with the following task command: +>task debug + +You may always use the task list command to see all available public tasks: +>task --list + WINDOWS: The taskfile should print on screen where the output library file was placed. It will be in the build directory usually under Debug or Release. diff --git a/lib/Utils.Memory/vnlib_mimalloc/package.json b/lib/Utils.Memory/vnlib_mimalloc/package.json index add660c..fb6ef71 100644 --- a/lib/Utils.Memory/vnlib_mimalloc/package.json +++ b/lib/Utils.Memory/vnlib_mimalloc/package.json @@ -1,10 +1,10 @@ { "name": "vnlib_mimalloc", - "version": "0.1.0", + "version": "0.0.0", "output_dir": "bin", "author": "Vaughn Nugent", "description": "A cmake cross-platform C89, NativeHeapApi compliant, mimalloc dynamic library. This project includes pre-build Windows x64 binaries as well as the source code for building the entire dynamic library on your system", - "copyright": "Copyright \u00A9 2023 Vaughn Nugent", + "copyright": "Copyright \u00A9 2024 Vaughn Nugent", "company": "Vaughn Nugent", - "repository": "https://github.com/VnUgE/VNLib.Core/tree/main/lib/Utils.Memory/vnlib_mimalloc" + "repository": "https://github.com/VnUgE/VNLib.Core/tree/master/lib/Utils.Memory/vnlib_mimalloc" }
\ No newline at end of file diff --git a/lib/Utils.Memory/vnlib_rpmalloc/CMakeLists.txt b/lib/Utils.Memory/vnlib_rpmalloc/CMakeLists.txt index fedde80..f3a2b6a 100644 --- a/lib/Utils.Memory/vnlib_rpmalloc/CMakeLists.txt +++ b/lib/Utils.Memory/vnlib_rpmalloc/CMakeLists.txt @@ -13,12 +13,11 @@ message(STATUS "Build type is '${build_type}'") set(CMAKE_C_STANDARD 11) #c11 is required for rpmalloc static assertions set(CMAKE_C_STANDARD_REQUIRED ON) -#export header files to the main project -file(GLOB HEADERS *.h) - #Add indepednent source files to the project set(VNLIB_RPMALLOC_SOURCES + "NativeHeapApi.h" "vnlib_rpmalloc.c" + "vendor/rpmalloc.h" "vendor/rpmalloc.c" ) @@ -26,8 +25,8 @@ set(VNLIB_RPMALLOC_SOURCES include_directories(vendor) #create shared/static libs -add_library(${_RP_PROJ_NAME} SHARED ${VNLIB_RPMALLOC_SOURCES} ${HEADERS}) -add_library(${_RP_PROJ_NAME}_static STATIC ${VNLIB_RPMALLOC_SOURCES} ${HEADERS}) +add_library(${_RP_PROJ_NAME} SHARED ${VNLIB_RPMALLOC_SOURCES}) +add_library(${_RP_PROJ_NAME}_static STATIC ${VNLIB_RPMALLOC_SOURCES}) #enable fPIC for shared library set_target_properties(${_RP_PROJ_NAME} PROPERTIES POSITION_INDEPENDENT_CODE ON) diff --git a/lib/Utils.Memory/vnlib_rpmalloc/NativeHeapApi.h b/lib/Utils.Memory/vnlib_rpmalloc/NativeHeapApi.h index 1738d19..b54313a 100644 --- a/lib/Utils.Memory/vnlib_rpmalloc/NativeHeapApi.h +++ b/lib/Utils.Memory/vnlib_rpmalloc/NativeHeapApi.h @@ -51,7 +51,7 @@ #ifdef _P_IS_WINDOWS #define VNLIB_HEAP_API __declspec(dllimport) #else - #define VNLIB_HEAP_API + #define VNLIB_HEAP_API extern #endif /* _P_IS_WINDOWS */ #endif /* !VNLIB_EXPORTING */ #endif /* !VNLIB_EXPORT */ @@ -72,12 +72,9 @@ typedef enum HeapCreationFlags HEAP_CREATION_SUPPORTS_REALLOC = 0x08 } HeapCreationFlags; -#ifdef _P_IS_WINDOWS -typedef void* LPVOID; -#endif /* !WIN32 */ - /* The vnlib ERRNO type, integer/process dependent, -internally represented as a pointer */ + internally represented as a pointer +*/ typedef void* ERRNO; /* A pointer to a heap structure that was stored during heap creation */ diff --git a/lib/Utils.Memory/vnlib_rpmalloc/Taskfile.yaml b/lib/Utils.Memory/vnlib_rpmalloc/Taskfile.yaml index 98dd914..f1c74a0 100644 --- a/lib/Utils.Memory/vnlib_rpmalloc/Taskfile.yaml +++ b/lib/Utils.Memory/vnlib_rpmalloc/Taskfile.yaml @@ -9,68 +9,108 @@ version: '3' vars: - PROJECT_NAME: 'vnlib_rpmalloc' + DEP_NAME: 'rpmalloc' + PROJECT_NAME: 'vnlib_{{ .DEP_NAME }}' + ARTIFACTS_OUT: '{{ .PROJECT_DIR }}/{{ .BINARY_DIR }}' tasks: default: desc: "Builds the entire project from source code without using the VNBuild build system for target machines" cmds: - #build with defaults - #init cmake build with greedy enabled - - cmake -Bbuild/ -DCMAKE_BUILD_TYPE=Release -DENABLE_GREEDY=1 {{.CMAKE_ARGS}} - - cmake --build build/ --config Release - - cmd: echo "Your vnlib_rpmalloc library file can be found in '{{.USER_WORKING_DIR}}/build'" - silent: true - + #init cmake build with greedy enabled + - task: build-internal + vars: + BUILD_TYPE: "Release" + CLI_ARGS: "-DENABLE_GREEDY=1 {{ .CLI_ARGS }}" + + - cmd: echo "Your {{ .PROJECT_NAME }} library file can be found in '{{ .USER_WORKING_DIR }}/build'" + silent: true + + debug: + cmds: + - task: build-internal + vars: + BUILD_TYPE: "debug" + CLI_ARGS: "-DENABLE_GREEDY=1 {{ .CLI_ARGS }}" + build: + desc: 'DO NOT USE! This function is used internally during releases.' cmds: #init cmake build with greedy enabled - - cmake -B./build -DENABLE_GREEDY=1 - - cmake --build build/ --config Debug - - cmake --build build/ --config Release + - cmake -B./build -DENABLE_GREEDY=1 {{ .CLI_ARGS }} + - cmake --build build/ --config debug + - cmake --build build/ --config release + + build-internal: + internal: true + cmds: + #build with defaults + #init cmake build with greedy enabled + - cmake -Bbuild/ -DCMAKE_BUILD_TYPE={{ .BUILD_TYPE }} {{ .CLI_ARGS }} + - cmake --build build/ --config {{ .BUILD_TYPE }} postbuild_success: - vars: - #required files to include in tar - TAR_FILES: "license.txt readme.txt rpmalloc_license.txt" - cmds: #make bin dir - - cmd: powershell -Command "New-Item -Type Directory -Force -Path './bin'" + - cmd: powershell mkdir -Force '{{ .ARTIFACTS_OUT }}' ignore_error: true + + - task: pack_parallel - #get licenses for debug - - task: licenses - vars: { TARGET: './build/Debug' } - - - task: licenses - vars: { TARGET: './build/Release' } - - #dynamic debug lib - - cd build/Debug && tar -czf '../../bin/msvc-x64-debug-{{.PROJECT_NAME}}.tgz' . - #release dll - - cd build/Release && tar -czf '../../bin/msvc-x64-release-{{.PROJECT_NAME}}.tgz' . + pack_parallel: + internal: true + deps: + - task: pack_source - #source code - - task: pack_source + - task: pack_artifacts + vars: { BUILD_TYPE: 'debug' } + - task: pack_artifacts + vars: { BUILD_TYPE: 'release' } - licenses: + #packs binary artifacts for distribution + pack_artifacts: + internal: true + deps: + - task: pack_resources + vars: { TARGET: './build/{{ .BUILD_TYPE }}' } + cmds: + - cmd: cd build/{{ .BUILD_TYPE }} && tar -czf '{{ .ARTIFACTS_OUT }}/msvc-x64-{{ .BUILD_TYPE }}-{{ .PROJECT_NAME }}.tgz' . + + pack_resources: + internal: true cmds: - #add rpmalloc license to binary output - - powershell -Command "Copy-Item -Path vendor/license -Destination '{{.TARGET}}/rpmalloc_license.txt'" - #add my license file - - powershell -Command "Copy-Item -Path ./license -Destination '{{.TARGET}}/license.txt'" - #add readme file - - powershell -Command "Copy-Item -Path ./build.readme.txt -Destination '{{.TARGET}}/readme.txt'" + - powershell cp NativeHeapApi.h '{{ .TARGET }}/' + - powershell cp vendor/license '{{ .TARGET }}/{{ .DEP_NAME }}_license.txt' + - powershell cp license '{{ .TARGET }}/license.txt' + - powershell cp build.readme.txt '{{ .TARGET }}/readme.txt' pack_source: - dir: '{{.USER_WORKING_DIR}}' + internal: true + desc: "Packs up the project source code and creates a tarball in the builds binary directory" + vars: + TARGET_SOURCE: '{{ .PROJECT_DIR }}/{{ .BINARY_DIR }}/src.tgz' + SOURCE_FILES: + CMakeLists.txt + Taskfile.yaml + vendor + LICENSE + NativeHeapApi.h + '{{ .PROJECT_NAME }}.c' + cmds: - - powershell -Command "tar --exclude build/* --exclude bin/* --exclude vendor/build/* -czf 'bin/src.tgz' ." + #tar up the source + - tar -czf '{{ .TARGET_SOURCE }}' {{ .SOURCE_FILES | join " " }} clean: ignore_error: true cmds: - for: [ bin/, build/ ] - cmd: powershell Remove-Item -Recurse '{{.ITEM}}' -Force
\ No newline at end of file + cmd: powershell rm -Recurse -Force '{{.ITEM}}' + + #called by module for initializing the project, it's not internal but hidden + dev-init: + internal: false + cmds: + - task: debug +
\ No newline at end of file diff --git a/lib/Utils.Memory/vnlib_rpmalloc/build.readme.txt b/lib/Utils.Memory/vnlib_rpmalloc/build.readme.txt index 79a7091..a8d1ab5 100644 --- a/lib/Utils.Memory/vnlib_rpmalloc/build.readme.txt +++ b/lib/Utils.Memory/vnlib_rpmalloc/build.readme.txt @@ -28,6 +28,12 @@ BUILDING: 2. Navigate to directory containing the Taskfile.yaml file in the root 3. Run the default task: > task (yes literally just type "task" and hit enter if you installed Task gobally) +You may build a debug version with the following task command: +>task debug + +You may always use the task list command to see all available public tasks: +>task --list + WINDOWS: The taskfile should print on screen where the output library file was placed. It will be in the build directory usually under Debug or Release. diff --git a/lib/Utils.Memory/vnlib_rpmalloc/package.json b/lib/Utils.Memory/vnlib_rpmalloc/package.json index 5235262..9fbd64c 100644 --- a/lib/Utils.Memory/vnlib_rpmalloc/package.json +++ b/lib/Utils.Memory/vnlib_rpmalloc/package.json @@ -1,10 +1,10 @@ { "name": "vnlib_rpmalloc", - "version": "0.1.4", + "version": "0.0.0", "output_dir": "bin", "author": "Vaughn Nugent", "description": "A project to maintain an x64 Windows dynamic and static library for RpMalloc by Mattias Jansson for use with the NativeHeapApi to for VNLib.Utils native heap loading. This project includes pre-build Windows x64 binaries as well as the source code for building the dynamic and static libraries on your system", - "copyright": "Copyright \u00A9 2023 Vaughn Nugent", + "copyright": "Copyright \u00A9 2024 Vaughn Nugent", "company": "Vaughn Nugent", - "repository": "https://github.com/VnUgE/VNLib.Core/tree/main/lib/Utils.Memory/vnlib_rpmalloc" + "repository": "https://github.com/VnUgE/VNLib.Core/tree/master/lib/Utils.Memory/vnlib_rpmalloc" }
\ No newline at end of file diff --git a/lib/Utils/src/VNLib.Utils.csproj b/lib/Utils/src/VNLib.Utils.csproj index 83d245c..2847afa 100644 --- a/lib/Utils/src/VNLib.Utils.csproj +++ b/lib/Utils/src/VNLib.Utils.csproj @@ -23,7 +23,7 @@ <PackageTags>vnlib,utils,utilities,vnlib utils,utility</PackageTags> <Description>.NET/8.0 Utilities library for high-performance common operations. Utilities and abstractions for building and diagnosing native memory implementations. Dyanmic native library loading, IO, extensions, data encoding, resource access, and asynchronous cooperation primitives.</Description> <PackageProjectUrl>https://www.vaughnnugent.com/resources/software/modules/VNLib.Core</PackageProjectUrl> - <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/main/lib/Utils</RepositoryUrl> + <RepositoryUrl>https://github.com/VnUgE/VNLib.Core/tree/master/lib/Utils</RepositoryUrl> <PackageReadmeFile>README.md</PackageReadmeFile> <PackageLicenseFile>LICENSE</PackageLicenseFile> <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance> diff --git a/third-party/DotNetCorePlugins/LICENSE.txt b/third-party/DotNetCorePlugins/LICENSE index c383288..c383288 100644 --- a/third-party/DotNetCorePlugins/LICENSE.txt +++ b/third-party/DotNetCorePlugins/LICENSE diff --git a/third-party/DotNetCorePlugins/Taskfile.yaml b/third-party/DotNetCorePlugins/Taskfile.yaml new file mode 100644 index 0000000..99839dc --- /dev/null +++ b/third-party/DotNetCorePlugins/Taskfile.yaml @@ -0,0 +1,49 @@ +# https://taskfile.dev + +#Called by the vnbuild system to produce builds for my website +#https://www.vaughnnugent.com/resources/software + +# Taskfile specific to DotNetCorePlugins project +# currently this taskfile only packs up the source code +# and test source code into a source code tgz file for +# distribution. The changes file and license file are also +# included + +version: '3' + +tasks: + + #when build succeeds, archive the output into a tgz + postbuild_success: + dir: '{{.USER_WORKING_DIR}}' + cmds: + - task: pack_source + + pack_source: + internal: true + dir: '{{ .USER_WORKING_DIR }}' + vars: + EXCLUDES: + --exclude='bin/*' + --exclude='obj/*' + #--exclude='.tarignore' + INCLUDES: + src/* + test/* + CHANGES.md + LICENSE + Taskfile.yaml + README.md + + cmds: + #pack up source code and put in output + - cmd: cd .. && tar {{ .EXCLUDES }} -czf '{{ .USER_WORKING_DIR }}/bin/src.tgz' {{ .INCLUDES }} + + +#Remove the output dirs on clean + clean: + dir: '{{.USER_WORKING_DIR}}' + ignore_error: true + cmds: + - for: [ bin/, obj/ ] + cmd: powershell -Command "rm -Recurse -Force '{{.ITEM}}'" diff --git a/third-party/DotNetCorePlugins/src/McMaster.NETCore.Plugins.csproj b/third-party/DotNetCorePlugins/src/McMaster.NETCore.Plugins.csproj index dcb24e7..03acbbc 100644 --- a/third-party/DotNetCorePlugins/src/McMaster.NETCore.Plugins.csproj +++ b/third-party/DotNetCorePlugins/src/McMaster.NETCore.Plugins.csproj @@ -1,16 +1,12 @@ <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> - <TargetFrameworks>net8.0</TargetFrameworks> + <TargetFramework>net8.0</TargetFramework> <OutputType>library</OutputType> <Nullable>enable</Nullable> <IsPublishable>False</IsPublishable> <GenerateDocumentationFile>true</GenerateDocumentationFile> - <PackageDescription>Provides API for dynamically loading assemblies into a .NET application. - -This package should be used by the host application which needs to load plugins. -See https://github.com/natemcmaster/DotNetCorePlugins/blob/main/README.md for more samples and documentation. - </PackageDescription> + <PackageDescription>Provides API for dynamically loading assemblies into a .NET application. This package should be used by the host application which needs to load plugins. See https://github.com/natemcmaster/DotNetCorePlugins/blob/main/README.md for more samples and documentation.</PackageDescription> <PackageTags>.NET Core;plugins</PackageTags> </PropertyGroup> @@ -23,4 +19,19 @@ See https://github.com/natemcmaster/DotNetCorePlugins/blob/main/README.md for mo <ProjectReference Include="..\..\..\lib\Utils\src\VNLib.Utils.csproj" /> </ItemGroup> + <ItemGroup> + <None Include="..\README.md"> + <Pack>False</Pack> + <PackagePath>\</PackagePath> + </None> + <None Include="..\LICENSE"> + <Pack>True</Pack> + <PackagePath>\</PackagePath> + <CopyToOutputDirectory>Always</CopyToOutputDirectory> + </None> + <None Include="..\Taskfile.yaml"> + <Pack>False</Pack> + </None> + </ItemGroup> + </Project> diff --git a/vnlib.core.build.sln b/vnlib.core.build.sln index 3463ebd..96ec416 100644 --- a/vnlib.core.build.sln +++ b/vnlib.core.build.sln @@ -46,6 +46,16 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Utils", "Utils", "{C8AB039C EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "VNLib.UtilsTests", "lib\Utils\tests\VNLib.UtilsTests.csproj", "{45EBB278-46D3-40D3-B358-0EBDC622F377}" EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "apps", "apps", "{7FEF3272-325A-4915-AC87-4300E48E0428}" +EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "VNLib.WebServer", "apps\VNLib.WebServer\src\VNLib.WebServer.csproj", "{1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}" +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "third-party", "third-party", "{1DABF62F-1321-4854-A365-17F17B1BB994}" +EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "McMaster.NETCore.Plugins", "third-party\DotNetCorePlugins\src\McMaster.NETCore.Plugins.csproj", "{5B330B55-D802-45FB-A09F-50654DC9442A}" +EndProject +Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "lib", "lib", "{2E02CD05-8D6C-48E3-AC86-B1DC8283B04B}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -224,15 +234,49 @@ Global {45EBB278-46D3-40D3-B358-0EBDC622F377}.Release|x64.Build.0 = Release|Any CPU {45EBB278-46D3-40D3-B358-0EBDC622F377}.Release|x86.ActiveCfg = Release|Any CPU {45EBB278-46D3-40D3-B358-0EBDC622F377}.Release|x86.Build.0 = Release|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Debug|x64.ActiveCfg = Debug|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Debug|x64.Build.0 = Debug|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Debug|x86.ActiveCfg = Debug|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Debug|x86.Build.0 = Debug|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Release|Any CPU.ActiveCfg = Release|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Release|x64.ActiveCfg = Release|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Release|x64.Build.0 = Release|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Release|x86.ActiveCfg = Release|Any CPU + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A}.Release|x86.Build.0 = Release|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Debug|x64.ActiveCfg = Debug|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Debug|x64.Build.0 = Debug|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Debug|x86.ActiveCfg = Debug|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Debug|x86.Build.0 = Debug|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Release|Any CPU.ActiveCfg = Release|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Release|x64.ActiveCfg = Release|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Release|x64.Build.0 = Release|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Release|x86.ActiveCfg = Release|Any CPU + {5B330B55-D802-45FB-A09F-50654DC9442A}.Release|x86.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE EndGlobalSection GlobalSection(NestedProjects) = preSolution + {A98E6166-C836-4EDE-9B0E-B5B3624EC8FE} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {5BB717C4-A2B8-493D-B9CA-8097FD3AB0A4} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {A88A4424-1537-4EA1-9B16-920CE450B83E} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {05E8CFD3-C678-4A75-8173-2E1C9F86C650} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {4F7FD27D-F7A9-4BFA-AE82-81C1BD27F6D5} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {46467E6F-9B97-47B2-B5D4-4CF74338682B} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {5C4699CB-DC05-42BC-81B5-B6ABEE1B7063} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {128A3DAE-7CDF-4075-A552-0FD8C9AFCCEC} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} {ED9B6918-1129-485F-9278-F9BCF13FE161} = {C8AB039C-E368-4C51-920D-06F2CEB2C95A} + {4091D0B2-B4F6-4BB4-A44E-44A128AD1628} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {68E4D3E2-AE19-403F-8436-1D87924B3903} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} + {A7EC3CF5-76B0-4235-AE9F-05409E3308B9} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} {4F7DCA8A-8B67-4277-8F14-FA43E479CB9B} = {A7EC3CF5-76B0-4235-AE9F-05409E3308B9} {E7A3F143-197B-49D2-BDF1-AC6FFE64C5AD} = {A7EC3CF5-76B0-4235-AE9F-05409E3308B9} + {C8AB039C-E368-4C51-920D-06F2CEB2C95A} = {2E02CD05-8D6C-48E3-AC86-B1DC8283B04B} {45EBB278-46D3-40D3-B358-0EBDC622F377} = {C8AB039C-E368-4C51-920D-06F2CEB2C95A} + {1C87FC71-9BC8-4FC1-8A99-9B9F373D1B3A} = {7FEF3272-325A-4915-AC87-4300E48E0428} + {5B330B55-D802-45FB-A09F-50654DC9442A} = {1DABF62F-1321-4854-A365-17F17B1BB994} EndGlobalSection GlobalSection(ExtensibilityGlobals) = postSolution SolutionGuid = {EA12D69B-8232-40F9-98F8-8A1C73796BFE} |