DevOps Steps in Cloud

Step 1: Container Image Build

1. In your source code repository (e.g., Git), include a Dockerfile that specifies how to build your application into a container image.

2. Configure your CI/CD tool (e.g., AWS CodeBuild, Jenkins) to build the Docker image using the Dockerfile. This can be done by executing a `docker build` command within your CI/CD script.

3. Ensure that the Docker image is built with the necessary dependencies and configurations.

Step 2: Container Registry

4. Choose a container registry service to store your Docker images. Common choices include:

   - AWS Elastic Container Registry (ECR) if you're using AWS.

   - Docker Hub for public images.

   - Other cloud providers' container registries (e.g., Google Container Registry, Azure Container Registry).

Step 3: Pushing Images

5. After building the Docker image, tag it with a version or unique identifier.

6. Use the `docker push` command to push the image to the selected container registry.

Step 4: Deployment

7. In your CD pipeline, integrate the deployment of the Docker container. This will depend on your application's architecture:

   - For Kubernetes-based applications, you can use Kubernetes manifests (YAML files) to define your application deployment. Update these manifests with the new Docker image version and apply them using `kubectl`.

   - For AWS-based applications, you can use services like Amazon ECS, Amazon EKS, or AWS Fargate for container orchestration. Update your task or service definition to use the new Docker image version.

Step 5: Automation and Rollback

8. Ensure that your CI/CD pipeline includes automation for container image tagging and deployment.

9. Implement rollback strategies, such as keeping the previous version of the container image in the registry to easily roll back in case of issues.

Step 6: Security

10. Pay attention to container image security. Scan container images for vulnerabilities using tools like Clair, Trivy, or AWS ECR image scanning.

11. Use container image signing and security policies to ensure that only trusted images are deployed.

Step 7: Monitoring

12. Implement monitoring and logging for your containerized application and infrastructure. Tools like Prometheus, Grafana, and cloud provider monitoring services can help.

Step 8: Integration with DevOps Pipeline

13. Integrate these container-related steps into your overall DevOps pipeline. For example, you can trigger the pipeline whenever changes are pushed to your Git repository.

Step 9: Documentation and Training

14. Ensure that your team is trained in containerization best practices and the use of your CI/CD pipeline.

With these steps, you can fully automate the build, registration (push), and deployment of Docker container images as part of your DevOps pipeline. This allows developers to focus on writing code, while the pipeline takes care of packaging and deploying applications consistently.

Delete Large Files From Your Git History Without Using Git LFS

 

If you want to delete large files from your Git history without using Git LFS, you can use the `git filter-branch` command along with the `--tree-filter` option to remove the files from your Git history. This process will rewrite the repository's history and remove the specified files.

Here's how you can do it:

1. Backup Your Repository:

   Before proceeding, make sure to create a backup of your repository to avoid data loss in case something goes wrong.

2. Identify Large Files:

   Identify the large files that you want to remove from the Git history, such as `data/hail-2015.csv`.

3. Run the `git filter-branch` Command:

   Use the `git filter-branch` command with the `--tree-filter` option to remove the large files from your Git history. Replace `data/hail-2015.csv` with the actual file path you want to remove.

   ```bash

   git filter-branch --force --index-filter \

   "git rm --cached --ignore-unmatch data/hail-2015.csv" \

   --prune-empty --tag-name-filter cat -- --all

   ```

This command will rewrite the Git history to exclude the specified file. Please note that this command will take some time to complete, especially for large repositories.

4. Clean Up Unreachable Objects:

   After running the `git filter-branch` command, there might be unreachable objects left in your repository. To remove them, run the following command:

   ```bash

   git reflog expire --expire=now --all && git gc --prune=now --aggressive

   ```

5. Force Push to Update Remote Repository:

   Since you've rewritten the Git history, you'll need to force push the changes to the remote repository:

   ```bash

   git push --force origin main

   ```

Replace `main` with the name of your branch if it's different.

Please use caution when performing these actions, especially if your repository is shared with others. Rewriting history can affect collaborators, so it's important to communicate with your team and coordinate this process.

Photo by Christina Morillo

Review Pull Request

 

Being leading diversified team for years. Besides Software Architect for more than 13 years for several different type of projects. I use to review several pull requests each day.

Sometimes it is tremendously time consuming. Especially when you have to tackle the not clear commit messages, SCRUM bug reports or work log are not clearly defined the issues or features.

Here is an example of an ML code review with a pull request from GitHub:

Title: Add a new model to the librar

Author: John Doe

Reviewer: Jane Doe

Description:

This pull request adds a new model to the library. The model is a simple linear regression model that can be used to predict house prices.

Changes:

* Added the new model to the library.

* Added unit tests for the new model.

* Updated the documentation to include the new model.

Review:

I have reviewed the pull request and I have found no major issues. The code is well-written and the unit tests are comprehensive. I recommend that the pull request be approved.

Approve/Reject:

Approved

Here are some additional things to keep in mind when reviewing ML code:

• Make sure that the code is well-documented. The documentation should be clear and concise, and it should explain how the code works.
• Test the code thoroughly. The unit tests should be comprehensive and they should cover all possible scenarios.
• Consider the impact of the changes. Think about how the changes will affect other parts of the code.
• Be constructive in your feedback. The goal of a code review is to help the author improve the code, not to criticize them.

Here are some tips on how to review pull requests:

1. Start by reading the commit messages. The commit messages should provide a clear and concise overview of the changes that have been made.
2. Run the code. This is the most important step in the review process. You should run the code to make sure that it compiles and runs without errors.
3. Check for style violations. The code should follow the project’s style guide. You can use a linter to help you check for style violations.
4. Test the code. You should write unit tests to test the code. This will help you to make sure that the code works as expected.
5. Review the documentation. If the code includes documentation, you should review the documentation to make sure that it is accurate and up-to-date.
6. Leave comments. If you find any problems with the code, you should leave comments on the pull request. This will help the author to fix the problems.
7. Approve or reject the pull request. Once you have reviewed the pull request, you should approve or reject it. If you approve the pull request, the code will be merged into the main branch. If you reject the pull request, the author will need to make changes to the code before it can be approved.
8. Be constructive in your feedback. The goal of a pull request review is to help the author improve the code, not to criticize them.
9. Be specific. Don’t just say “the code is bad.” Explain what is wrong with the code and how it can be improved.
10. Be helpful. If you can, offer suggestions on how the code can be improved.
11. Be timely. Don’t leave pull requests hanging for days or weeks.

There are a few ways to make the review test automatic as much as possible for eg. Python.

• Use a linter. A linter is a tool that can be used to check for style violations in Python code. There are many different linters available, such as flake8 and pylint.
• Write unit tests. Unit tests are small, isolated tests that can be used to verify the functionality of a piece of code. Unit tests can be run automatically, which can help to make the review process more efficient.
• Use a continuous integration (CI) server. A CI server is a tool that can be used to run unit tests and other automated checks on code changes. CI servers can be configured to run automatically whenever a new pull request is submitted.

Here are some specific tools and libraries that you can use to automate the review test for Python:

• Flake8: A linter that can be used to check for style violations in Python code.
• Pylint: A linter that can be used to check for style violations and potential bugs in Python code.
• Unittest: A library that can be used to write unit tests for Python code.
• Travis CI: A CI server that can be used to run unit tests and other automated checks on code changes.
• CircleCI: Another CI server that can be used to run unit tests and other automated checks on code changes.

Thank you.

Github

Pull Request

Pull

Development

Code Review