breezewiki-docs/docs.scrbl

#lang scribble/base

@(require scribble/manual scribble/core racket/list)
@(define (code . pre-content) (elem (apply literal pre-content) #:style 'tt))

@title{BreezeWiki Documentation}

@url{https://breezewiki.com}

@local-table-of-contents[]

@; -----------------------------------------------------------------------------

@section{Links}

@subsection{Official}

@tabular[#:style (style "user-table" null)
         (map (λ (row) (list (first row) (if (string? (second row)) (url (second row)) (second row))))
              `(("Official website" "https://breezewiki.com")
                ("Community" "https://matrix.to/#/#breezewiki:cadence.moe")
                ("How to bug report or feature request" ,(Secref "Reporting Bugs"))
                ("Planned features" "https://todo.sr.ht/~cadence/breezewiki-todo")
                ("Source code" "https://gitdab.com/cadence/breezewiki")
                ("Documentation source" "https://gitdab.com/cadence/breezewiki-docs")
                ))]

@subsection{Instances}

@tabular[#:style (style "user-table instance-list user-table--title" null)
         (map (λ (row) (list (if (string? (second row)) (hyperlink (second row) (first row)) (first row))
                             (if (string? (third row)) (url (third row)) (third row))))
              `((,(literal "Hoster") #f ,(literal "Link"))
                ("Cadence" "https://cadence.moe" "https://breezewiki.com")
                ("PussTheCat.org" "https://pussthecat.org" "https://breezewiki.pussthecat.org")
                ("Odyssey346" "https://odyssey346.dev" "https://bw.odyssey346.dev")
                ("~vern" "https://vern.cc" "https://bw.vern.cc")
                ("Esmail" "https://en.esmailelbob.xyz" "https://breezewiki.esmailelbob.xyz")
                ))]

@subsection[#:tag "Tools"]{Tools}

@subsubsection{Cadence's redirector}

@url{https://docs.breezewiki.com/files/redirector.user.js}

Displays a giant banner and you can click a link to go to BreezeWiki.

@subsubsection{Barrow's redirector}

@url{http://alphamethyl.barr0w.net/~barrow/src/barrownet_ca_redirector/breezewiki_redirector.user.js}

Just sends you to BreezeWiki automatically.

@; -----------------------------------------------------------------------------

@section{Automatic Redirection}

If you'd like to be automatically redirected from fandom.com to breezewiki.com, follow @italic{one} of the methods below.

@subsection{Browser extension: Redirector}

This will redirect you with no fanfare.

@itemlist[#:style 'ordered
@item{Install the extension: @hyperlink["https://addons.mozilla.org/en-GB/firefox/addon/redirector/" "Firefox"] or @hyperlink["https://chrome.google.com/webstore/detail/redirector/ocgpenflpmgnfapjedencafcfakcekcd" "Chrome"]}
@item{Download this file: @url{https://docs.breezewiki.com/files/breezewiki-redirector.json}}
@item{Click on the Redirector extension icon, click @italic{Edit Redirects}, click @italic{Import}, then choose the @italic{breezewiki-redirector.json} file you just downloaded. It will say successfully imported.}
@item{Once it's imported, you can delete the @italic{breezewiki-redirector.json} file from your computer.}
]

@subsection{Browser extension: Violentmonkey}

@nested[#:style (style "block-note" null)]{Violentmonkey executes @italic{userscripts} created by other people, which can do many things to your web browser, more than just redirect you. Some may be helpful, some may be harmful. Make sure you only install userscripts that you trust.}

@itemlist[#:style 'ordered
@item{@hyperlink["https://violentmonkey.github.io/get-it/" "Install the extension"] (or you can choose another userscript manager, like Greasemonkey or Tampermonkey)}
@item{Look at the @Secref["Tools"] section of this website, choose somebody's userscript, and click the link.}
@item{Violentmonkey will show you the code of the userscript, and offer to install it for you. If you want to proceed with the installation, click @italic{Confirm Installation}.}
]

@; -----------------------------------------------------------------------------

@section[#:tag "Reporting Bugs"]{Reporting Bugs}

Bug reports are important, and so are feature requests! Thank you for your interest. Here's how to send in yours.

There is the @italic{mailing list} which is the first line for people like you to report to. Once a bug is confirmed, I will copy it onto the @italic{todo tracker} as a task item.

Please spend one minute @hyperlink["https://todo.sr.ht/~cadence/breezewiki-todo" "checking the todo tracker"] so you don't submit a duplicate of somebody else's report.

For real-time chat with the community, check out @url{https://matrix.to/#/#breezewiki:cadence.moe}.

@subsection{Actually sending a report}

To actually send in your report, compose an email to @hyperlink["mailto:~cadence/breezewiki-discuss@lists.sr.ht" "~cadence/breezewiki-discuss@lists.sr.ht"] with a useful subject line and a description.

Please make sure to include the following information:

@itemlist[
@item{Which URL can I visit to see it for myself?}
@item{Which part of the page are you talking about?}
@item{Does it happen every time?}
]

Thanks!

@; -----------------------------------------------------------------------------

@section{Running}

@subsection{Running a compiled executable}

This method is recommended for people who do not want to edit the source code.

The distribution includes the Racket runtime, libraries, and everything else needed to run BreezeWiki.

@itemlist[#:style 'ordered
@item{Download the distribution: @url{https://docs.breezewiki.com/files/breezewiki-dist.tar.gz}}
@item{Unpack the archive.}
@item{Execute the binary @code{breezewiki-dist/bin/dist}.}
]

@subsection{Running the source code}

This method is recommended for people who want to edit the source code, or for people who can't use the compiled distribution.

@itemlist[#:style 'ordered
@item{Install Racket. Please install the latest version. 8.4 or later is required.}
@item{Clone the repository from @url{https://gitdab.com/cadence/breezewiki}.}
@item{Run @code{raco pkg install --auto --skip-installed} in the repository to install dependencies.}
]

@nested[#:style (style "block-note" null)]{If building in an automated environment, like CI, use the flags @code{--batch --auto --no-docs --skip-installed}.}

Now you can start BreezeWiki using one of the following methods.

@itemlist[
@item{Run @code{racket dist.rkt} to start BreezeWiki.}
@item{Evaluate @code{breezewiki.rkt} in DrRacket or in the Racket REPL to start BreezeWiki with module hot-reloading. Helpful for development!}
]

To be able to start up BreezeWiki faster, use @code{raco make dist.rkt} to byte-compile individual files.

@subsection{Running with Docker}

@nested[#:style (style "block-note" null)]{There is no official Docker support. Information herein is created by members of the community. Using Docker makes it more difficult to debug and find help if you encounter problems.}

Image provided by PussTheCat.org: @url{https://github.com/PussTheCat-org/docker-breezewiki-quay}

@subsection{What next?}

If you're running BreezeWiki in production (i.e. not for development) then check out the @secref["Required options"] section of the configuration page.

@; -----------------------------------------------------------------------------

@section[#:tag "Configuration"]{Configuration}

@subsection{File}

Configuration options are added to the @code{config.ini} file. It is a regular INI file.

If you do not specify a specific option, the internal default will be used. If the file is missing or empty, all defaults will be used.

If you're using a compiled distribution of BreezeWiki, the main @code{config.ini} file will actually be a symlink to the real location of the file. Make sure not to erase the symlink, or your settings will be ignored!

@subsubsection{Example}

Here is an example configuration file. It shows all the available settings and all their defaults.
@verbatim{
canonical_origin =
debug = false
log_outgoing = true
port = 10416
strict_proxy = true
}

@subsubsection{Format}

@itemlist[
@item{A line with an equals sign @code{=} defines one setting.}
@item{The text before the equals sign is the @italic{key}.}
@item{The text after the equals sign is the @italic{value}. It may be empty.}
@item{Quote marks are optional - they are never necessary.}
@item{Spacing around the equals sign is optional.}
@item{@italic{False} values are represented by either empty space, or the text @code{false}. Anything else is @italic{True}.}
]

@subsection{Environment variables}

By popular demand, environment variables can be used as an alternative to the configuration file. The configuration file will be read first and will override the default settings, then, if any environment variables are present, they will override the values in the configuration files.

Environment variables start with @code{bw_} and then the name of the setting. They can be uppercase or lowercase.

@subsubsection{Example}

Here is an example of using environment variables in a typical shell:

@verbatim{
bw_port=4000 bw_debug=false racket dist.rkt
}

@subsection[#:tag "Required options"]{Required options}

No options are strictly required, but some are highly recommended, depending on where you're running BreezeWiki.

When running for local development, @code{debug = true} is recommended.

When running in production, @code{canonical_origin} is highly recommended.

@subsection{Detailed Options}

@defthing[canonical_origin value #:value ""]{
The URL that the homepage of the instance is intended to be accessed at.

For example, @code{https://breezewiki.com}
}

@defthing[debug boolean #:value "false"]{
Enables debugging mode, for debugging BreezeWiki during development.

Enables more runtime checks and more verbose output. Turns off some browser caching.
}

@defthing[log_outgoing boolean #:value "true"]{
Whether to log outgoing requests to Fandom to the console.
}

@defthing[port value #:value "10416"]{
Which port to run the server on.
}

@defthing[strict_proxy boolean #:value "true"]{
Whether to put more URLs through the proxy. If false, just a minimal set is proxied. If true, additionally proxies page stylesheets and links to image files, thereby reducing the potential for end-users to connect to Fandom servers.
}

@; -----------------------------------------------------------------------------

@section{Developing BreezeWiki}

@subsection{Racket}

BreezeWiki is programmed in the @hyperlink["https://racket-lang.org" "Racket"] programming language.

If you are new to programming entirely, BreezeWiki is likely not a good introductory project for you. I recommend purchasing a physical book about programming targeted at beginners. (In 2022, internet searches for beginner programming questiosn are sadly filled with low quality results or straight up misinformation.)

If you already know programming concepts but are new to Racket, I recommend reading @hyperlink["https://docs.racket-lang.org/quick/" "Quick: An Introduction to Racket with Pictures"] and trying the instructions for yourself. This should fill you in on the basics of practically using the Racket language. After reading that, if you want to know the fundamentals even more in-depth, you can check out the @hyperlink["https://docs.racket-lang.org/guide/index.html" "Racket Guide"], which is better to jump around in rather than read from start to end.

I @italic{highly recommend} using the official DrRacket IDE to write Racket code, particularly if you are a beginner. It applies useful indentation automatically and it has extremely good hover hints. Once you are familiar with Racket, you could configure Emacs, VSCode, Vim, or your favourite other editor to understand Racket's style rules, though you might still miss out on the really good hover hints.

@subsection{Files}

@code{breezewiki.rkt} and @code{dist.rkt} are entrypoints. They do as little as possible, just requiring the page functions and starting the web server.

@code{src/page-*.rkt} contains the page functions. Each file has instructions on how to render a specific kind of page. For example, @code{src/page-wiki.rkt} renders the usual wiki pages by contacting Fandom servers, altering the HTML tree, and sending the response to the browser.

Everything else in @code{src/} is a utility file that is required in by the pages as needed.

@subsection{Fandom endpoints}

BreezeWiki mostly uses the MediaWiki APIs rather than scraping full pages pages. If I need to add functionality in the future that can't be covered through the APIs, I would consider changing to scraping full pages.

@subsection{HTML tree transformations}

@hyperlink["https://www.mediawiki.org/wiki/API:Parsing_wikitext" "MediaWiki's Parse API"] returns the contents of a page as HTML, but returning this wholesale isn't good enough. Several modifications need to be made to the page before it's suitable for display on BreezeWiki, such as altering links to have the correct prefix, making all images visible without JavaScript, and enclosing tables to allow horizontal scrolling.

To do these transformations, the @code{update-tree} function recurses through the whole HTML tree from MediaWiki, executing a function on each element. This function returns the new element to replace with, allowing update-tree to eventually build a whole brand new tree. If the function decides that there's nothing to be done for a particular element, it just returns the same element. You can see the largest example of this in @code{src/page-wiki.rkt}, where many transformations can be applied to every element.

Background: Racket is a dialect of Lisp, a class of programming languages that are good at manipulating lists of symbols. XML and HTML trees are represented as @italic{X-expressions}, which you can read a bit more about within the @hyperlink["https://docs.racket-lang.org/pollen/second-tutorial.html#%28part._.X-expressions%29" "Pollen documentation: X-expressions"]. (The rest of the Pollen documentation is unrelated.) An X-expression is often called an xexp or an xexpr for short. BreezeWiki includes @code{src/xexpr-utils.rkt} for some helpful functions that query and manipulate X-expressions.

@subsection{Programming}

If @code{breezewiki.rkt} is launched (in a REPL) it will dynamically load in the pages and watch them and their dependencies for changes. If you edit and save a file within @code{src/} it will likely be reloaded automatically.

Creating a new page requires editing @code{src/dispatcher-tree.rkt} to define which URLs the page should appear from, and @code{breezewiki.rkt} and @code{dist.rkt} to cause the page to actually be loaded. The first time you create a page it won't be loaded automatically and you'll need to restart BreezeWiki.

Automatic reloading helps with rapid development and avoiding long process startup times. But if the long startup times still frustrate you, you can use @code{raco make breezewiki.rkt dist.rkt} to byte-compile files for faster startup.

@subsection{Testing}

Most files also contain unit tests, within the @code["(module+ test"] sections. Evaluating a file within either DrRacket or Emacs-racket-mode automatically runs the tests in that file. Since almost all functions are pure functions (i.e. operate based on their inputs and outputs rather than relying on application state) they are easy to unit test.

You can run all tests in all files with @code["raco test --direct (grep -l 'module+ test' (git ls-files))"] which is helpful to check for regressions before committing.