Auth0 上で freeCodeCamp 開発用のテストテナントを使用しているため、カスタムドメインを設定することはできません。 そのため、すべてのリダイレクトコールバックとログインページが `https://freecodecamp-dev.auth0.com/` のようなデフォルトドメインに表示されます。 This does not affect the functionality and is as close to production as we can get.
pm2 start ./lib/production-start.js -i max --max-memory-restart 600M --name org
```
### Logging and Monitoring
```console
pm2 logs
```
```console
pm2 monit
```
### Updating Instances (Maintenance)
Code changes need to be deployed to the API instances from time to time. It can be a rolling update or a manual update. The later is essential when changing dependencies or adding environment variables.
> [!ATTENTION] The automated pipelines are not handling dependencies updates at the minute. We need to do a manual update before any deployment pipeline runs.
#### 1. Manual Updates - Used for updating dependencies, env variables.
Code changes need to be deployed to the API instances from time to time. It can be a rolling update or a manual update. The later is essential when changing dependencies or adding environment variables.
> [!ATTENTION] The automated pipelines are not handling dependencies updates at the minute. We need to do a manual update before any deployment pipeline runs.
#### 1. Manual Updates - Used for updating dependencies, env variables.
1. Stop all instances
```console
pm2 stop all
```
2. Install or update dependencies
3. Start Instances
```console
pm2 start all --update-env && pm2 logs
```
#### 2. Rolling updates - Used for logical changes to code.
```console
pm2 reload all --update-env && pm2 logs
```
> [!NOTE] We are handling rolling updates to code, logic, via pipelines. You should not need to run these commands. These are here for documentation.
## Work on Chat Servers
Our chat servers are available with a HA configuration [recommended in Rocket.Chat docs](https://docs.rocket.chat/installation/docker-containers/high-availability-install). The `docker-compose` file for this is [available here](https://github.com/freeCodeCamp/chat-config).
We provision redundant NGINX instances which are themselves load balanced (Azure Load Balancer) in front of the Rocket.Chat cluster. The NGINX configuration file are [available here](https://github.com/freeCodeCamp/chat-nginx-config).
2. Configure the required environment variables and instance IP addresses.
3. Run rocket-chat server
```console
docker-compose config
docker-compose up -d
```
### Logging and Monitoring
1. Check status for NGINX service using the below command:
```console
sudo systemctl status nginx
```
2. Check status for running docker instances with:
```console
docker ps
```
### Updating Instances (Maintenance)
**NGINX Cluster:**
Config changes to our NGINX instances are maintained on GitHub, these should be deployed on each instance like so:
1. SSH into the instance and enter sudo
```console
sudo su
```
2. Get the latest config code.
```console
cd /etc/nginx
git fetch --all --prune
git reset --hard origin/main
```
3. Test and reload the config [with Signals](https://docs.nginx.com/nginx/admin-guide/basic-functionality/runtime-control/#controlling-nginx).
```console
nginx -t
nginx -s reload
```
**Docker Cluster:**
1. SSH into the instance and navigate to the chat config path
```console
cd ~/chat
```
2. Get the latest config code.
```console
git fetch --all --prune
git reset --hard origin/main
```
3. Pull down the latest docker image for Rocket.Chat
```console
docker-compose pull
```
4. Update the running instances
```console
docker-compose up -d
```
5. Validate the instances are up
```console
docker ps
```
6. Cleanup extraneous resources
```console
docker system prune --volumes
```
Output:
```console
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all volumes not used by at least one container
- all dangling images
- all dangling build cache
Are you sure you want to continue? [y/N] y
```
Select yes (y) to remove everything that is not in use. This will remove all stopped containers, all networks and volumes not used by at least one container, and all dangling images and build caches.
> [!ATTENTION] For client applications, the shell script can't be resurrected between Node.js versions with `pm2 resurrect`. Deploy processes from scratch instead. This should become nicer when we move to a docker based setup.
>
> If using PM2 for processes you would also need to bring up the applications and save the process list for automatic recovery on restarts.
Get the uninstall instructions/commands with the `unstartup` command and use the output to remove the systemctl services
```console
pm2 unstartup
```
Get the install instructions/commands with the `startup` command and use the output to add the systemctl services
Quick commands for PM2 to list, resurrect saved processes, etc.
```console
pm2 ls
```
```console
pm2 resurrect
```
```console
pm2 save
```
```console
pm2 logs
```
## Installing and Updating Azure Pipeline Agents
See: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=azure-devops and follow the instructions to stop, remove and reinstall agents. Broadly you can follow the steps listed here.
You would need a PAT, that you can grab from here: https://dev.azure.com/freeCodeCamp-org/_usersSettings/tokens
### Installing agents on Deployment targets
Navigate to [Azure Devops](https://dev.azure.com/freeCodeCamp-org) and register the agent from scratch in the requisite [deployment groups](https://dev.azure.com/freeCodeCamp-org/freeCodeCamp/_machinegroup).
> [!NOTE] You should run the scripts in the home directory, and make sure no other `azagent` directory exists.
### Updating agents
Currently updating agents requires them to be removed and reconfigured. This is required for them to correctly pick up `PATH` values and other system environment variables. We need to do this for instance updating Node.js on our deployment target VMs.
1. Navigate and check status of the service
```console
cd ~/azagent
sudo ./svc.sh status
```
2. Stop the service
```console
sudo ./svc.sh stop
```
3. Uninstall the service
```console
sudo ./svc.sh uninstall
```
4. Remove the agent from the pipeline pool
```console
./config.sh remove
```
5. Remove the config files
```console
cd ~
rm -rf ~/azagent
```
Once You have completed the steps above, you can repeat the same steps as installing the agent.
# Flight Manual - Email Blast
We use [a CLI tool](https://github.com/freecodecamp/sendgrid-email-blast) to send out the weekly newsletter. To spin this up and begin the process:
1. Sign in to DigitalOcean, and spin up new droplets under the `Sendgrid` project. Use the Ubuntu Sendgrid snapshot with the most recent date. This comes pre-loaded with the CLI tool and the script to fetch emails from the database. With the current volume, three droplets are sufficient to send the emails in a timely manner.
2. Set up the script to fetch the email list.
```console
cd /home/freecodecamp/scripts/emails
cp sample.env .env
```
You will need to replace the placeholder values in the `.env` file with your credentials.
3. Run the script.
```console
node get-emails.js emails.csv
```
This will save the email list in an `emails.csv` file.
4. Break the emails down into multiple files, depending on the number of droplets you need. This is easiest to do by using `scp` to pull the email list locally and using your preferred text editor to split them into multiple files. Each file will need the `email,unsubscribeId` header.
5. Switch to the CLI directory with `cd /home/sendgrid-email-blast` and configure the tool [per the documentation](https://github.com/freeCodeCamp/sendgrid-email-blast/blob/main/README.md).
6. Run the tool to send the emails, following the [usage documentation](https://github.com/freeCodeCamp/sendgrid-email-blast/blob/main/docs/cli-steps.md).
7. When the email blast is complete, verify that no emails have failed before destroying the droplets.
We use a custom [theme](https://github.com/freeCodeCamp/news-theme) for our news publication. Adding the following changes to the theme enables the addition of new languages.
1. Include an `else if` statement for the new [ISO language code](https://www.loc.gov/standards/iso639-2/php/code_list.php) in [`setup-locale.js`](https://github.com/freeCodeCamp/news-theme/blob/main/assets/config/setup-locale.js)
2. Create an initial config folder by duplicating the [`assets/config/en`](https://github.com/freeCodeCamp/news-theme/tree/main/assets/config/en) folder and changing its name to the new language code. (`en` —> `es` for Spanish)
3. Inside the new language folder, change the variable names in `main.js` and `footer.js` to the relevant language short code (`enMain` —> `esMain` for Spanish)
4. Duplicate the [`locales/en.json`](https://github.com/freeCodeCamp/news-theme/blob/main/locales/en.json) and rename it to the new language code.
5. In [`partials/i18n.hbs`](https://github.com/freeCodeCamp/news-theme/blob/main/partials/i18n.hbs), add scripts for the newly created config files.
6. Add the related language `day.js` script from [cdnjs](https://cdnjs.com/libraries/dayjs/1.10.4) to the [freeCodeCamp CDN](https://github.com/freeCodeCamp/cdn/tree/main/build/news-assets/dayjs/1.10.4/locale)
Update the publication assets by going to the Ghost dashboard > settings > general and uploading the publications's [icon](https://github.com/freeCodeCamp/design-style-guide/blob/master/assets/fcc-puck-500-favicon.png), [logo](https://github.com/freeCodeCamp/design-style-guide/blob/master/downloads/fcc_primary_large.png), and [cover](https://github.com/freeCodeCamp/design-style-guide/blob/master/assets/fcc_ghost_publication_cover.png).
