Reading time 5 min

Peter Jans

Like every year, Amazon held its AWS re:Invent 2021 in Las Vegas. While we weren’t able to attend in person due to the pandemic, as an AWS Partner we were eager to follow the digital event. Below is a quick rundown of our highlights of the event to give you a summary in case you missed it! AWS closer to home AWS will build 30 new ‘ Local Zones ’ in 2022, including one in our home base: Belgium. AWS Local Zones are a type of infrastructure deployment that places compute, storage, database, and other select AWS services close to large population and industry centers. The Belgian Local Zone should be operational by 2023. Additionally, the possibilities of AWS Outposts have increased . The most important change is that you can now run far more services on your own server delivered by AWS. Quick recap: AWS Outposts is a family of fully managed solutions delivering AWS infrastructure and services to virtually any on-premises or edge location for a consistent hybrid experience. Outposts was previously only available in a 42U Outposts rack configuration. From now on, AWS offers a variety of form factors, including 1U and 2U Outposts servers for when there’s less space available. We’re very tempted to get one for the office… AWS EKS Anywhere was previously announced, but is now a reality! With this service, it’s possible to set up a Kubernetes cluster on your own infrastructure or infrastructure from your favorite cloud provider, while still managing it through AWS EKS. All the benefits of freedom of choice combined with the unified overview and dashboard of AWS EKS. Who said you can’t have your cake and eat it too? Low-code to regain primary focus With Amplify Studio , AWS takes the next step in low-code development. Amplify Studio is a fully-fledged low-code generator platform that builds upon the existing Amplify framework. The platform allows users to build applications through drag and drop with the possibility of adding custom code wherever necessary. Definitely something we’ll be looking at on our next Ship-IT Day! Machine Learning going strong(er) Ever wanted to start with machine learning, but not quite ready to invest some of your hard-earned money? With SageMaker Studio Lab , AWS announced a free platform that lets users start exploring AI/ML tools without having to register for an AWS account or leave credit card details behind. You can try it yourself for free in your browser through Jupyter notebooks ! Additionally, AWS announced SageMaker Canvas : a visual, no-code machine learning capability for business analysts. This allows them to get started with ML without having extensive experience and get more insights in data. The third chapter in the SageMaker saga consists of SageMaker Ground Truth Plus . With this new service, you hire a team of experts to train and label your data, a traditionally very labor intensive process. According to Amazon, customers can expect to save up to 40% through SageMaker Ground Truth Plus. There were two more minor announcements: the AI ML Scholarschip Program , a free program for students to get to know ML tools, and Lex Automated Chatbot Designer , which lets you quickly develop a smart chatbot with advanced natural language processing support. Networking for everyone Tired of less than optimal reception or a slow connection? Why not build your own private 5G network? Yep: with AWS Private 5G , Amazon delivers the hardware, management and sim cards for you to set up your very own 5G network. Use cases (besides being fed up with your current cellular network) include warehouses or large sites (e.g. a football stadium) that require low latency, excellent coverage and a large bandwidth. The best part? Customers only pay for the end user’s usage of the network. Continuing the network theme, there’s now AWS Cloud WAN . This service allows users to build a managed WAN (Wide Area Network) to connect cloud and on-premise environments with a central management UI on a network components level as well as service level. Lastly, there’s also AWS Workspaces Web . Through this service, customers can grant employees safe access to internal website and SaaS applications. The big advantage here is that information critical to the company never leaves the environment and doesn’t leave any traces on workstations, thanks to a non-persistent web browser. Kubernetes anyone? No AWS event goes without mentioning Kubernetes, and AWS re:Invent 2021 is no different. Amazon announced two new services in the Kubernetes space: AWS Karpenter and AWS Marketplace for Containers Anywhere . With AWS Karpenter, managing autoscaling Kubernetes infrastructure becomes both simpler and less restrictive. It takes care of automatically starting compute when the load of an application changes. Interestingly, Karpenter is fully open-source, a trend which we’ll see more and more according to Amazon. AwS Marketplace for Containers Anywhere is primarily useful for customers who’ve already fully committed to container managed platforms. It allows users to search, subscribe and deploy 3rd party Kubernetes apps from the AWS Marketplace in any Kubernetes cluster, no matter the environment. IoT updates There have been numerous smaller updates to AWS’s IoT services, most notably to: GreenGrass SSM , which now allows you to securely manage your devices using AWS Systems Manager Amazon Monitron to predict when maintenance is required for rotating parts in machines AWS IoT TwinMaker , to simply make Digital Twins of real-world systems AWS IoT FleetWise , whichs helps users to collect vehicle data in the cloud in near-real time. Upping the serverless game In the serverless landscape, AWS announced serverless Redshift , EMR , MSK , and Kinesis . This enables to set up services while the right instance type is automatically linked. If the service is not in use, the instance automatically stops. This way, customers only pay for when a service is actually being used. This is particularly interesting for experimental services and integrations in environments which do not get used very often. Sustainability Just like ACA Group’s commitment to sustainability , AWS is serious about their ambition towards net-zero carbon by 2040. They’ve developed the AWS Customer Carbon Footprint tool, which lets users calculate carbon emissions through their website . Other announcements included AWS Mainframe Modernization , a collection of tools and guides to take over existing mainframes with AWS, and AWS Well-Architected Framework , a set of design principles, guidelines, best practices and improvements to validate sustainability goals and create reports. We can't wait to start experimenting with all the new additions and improvements announced at AWS re:Invent 2021. Thanks for reading! Discover our cloud hosting services

Reading time 3 min

Peter Jans

The world is rapidly changing, both from a technological and environmental point of view. Often, these challenges go hand in hand. For example, through the push towards electric vehicles, smart homes and sustainable energy. But while there has been a longstanding focus on the automotive, manufacturing and agricultural industries, there is no pathway to a cleaner environment without addressing the sizable energy consumption of data centers and cloud computing. The carbon footprint of cloud computing According to the International Energy Agency’s (IEA) latest report , data centers around the world in 2021 used 220 to 320 TWh of electricity, which is around 0.9 to 1.3% of the global electricity demand. In addition, global data transmission networks consumed 260-340 TWh, or 1.1 to 1.4% of electricity. Combined, data centers and transmission networks contribute to 0.9% of energy-related emissions. While these may seem fairly low numbers, the demand for data services is rising exponentially. Global internet traffic surged over the past decade, an evolution that accelerated during the pandemic. Since 2010, the number of internet users across the world has more than doubled and global internet traffic has increased 15-fold , or 30% per year . This means that the carbon footprint of cloud computing is something all companies, large or small, must consider. But what can you do without sacrificing the computing power needed to support innovation and deliver goods and services as promised? Amazon Web Services (AWS) While cloud computing also comes with a footprint, it offers a much more eco-friendly way to operate your IT systems than local servers. That’s why we believe a cloud-first approach is key to make your business more sustainable. Especially when cloud-based technologies are powered with renewable energy. That’s why ACA Group carefully chooses its partnerships and evaluates the environmental impact of those partners. In this context, we have selected AWS as a cloud provider. Combined with our flexible Kubernetes setups, it allows us to choose for the least amount of carbon emissions while still meeting (and even exceeding) the expectations of our customers. It shows that cloud computing needs do not come at the planet’s expense. But why AWS? As the world’s most prominent cloud provider, Amazon Web Services is focused on efficiency and continuous innovation across its global infrastructure. In fact, they are well on their way to powering their operations with 100% renewable energy by 2025. Amazon recently became the world’s largest corporate purchaser of renewable energy ; Their investments supply enough electricity to power 3 million US households for a year. Efficient computing Creating clean energy sources is essential, but no less important is rethinking how computing resources are allocated. In a cloud efficiency report, 451 Research showed that AWS’s infrastructure is 3.6 times more energy efficient than the median of U.S. enterprise data centers they surveyed. Amazon attributes this greater efficiency to, among other things, removing the central uninterruptible power supply from their data design and integrating small battery packs and custom power supplies into the server racks. Tese changes combined reduce energy conversion loss by about 35%. The servers themselves are more efficient as well: their Graviton2 CPUs are extremely power- efficient and offer better performance per watt than any other processor currently in use in Amazon data centers. AWS offers unlimited access to cloud computing and services. While this comes at a price, efficient use of resources not only reduces costs, but also indirectly reduces carbon emissions. How can you achieve this? Build applications that are resource-efficient. Consume resources with the lowest possible footprint. Maximize the output on resources used. Reduce the amount of data and distance traveled across the network. Use resources just-in-time. ➡️ Curious how we at ACA Group set up our cloud stacks for maximum sustainability without giving up power, availability and flexibility? Talk to us here !

Reading time 8 min

Bregt Coenen

In the fast-moving world of IT, ACA Group constantly takes the time to explore innovative solutions and tools to provide the best services to our customers. Recently, we shared our experience with Flux, a CloudNative Continuous Deployment tool that implements GitOps. What is Harbor? Harbor is a CloudNative tool designed to leverage the flexibility, scalability and resilience of the cloud. It is a containerized solution that provides advanced features such as vulnerability scanning and artifact registry. Harbor can run on any Kubernetes based solution on public/private cloud, but also on your local Kubernetes cluster. It is a self-managed solution that needs to be deployed on your Kubernetes cluster. You can choose the components to deploy based on your needs. For some features, for example vulnerability scanning, a selection between different tools can be made. The benefits of Harbor Registery We are excited to use Harbor Registry for our projects in the coming months because of the following benefits: Harbor is easy to scale : All components can be set up with multiple replicas, preventing unexpected downtime and providing fail-over. This ensures that your container images are always available when you need them. Harbor is composable : It allows you to deploy only the workloads for the features you use. Harbor is not only a contaner registery : It can, for example, also be used as a Chartmuseum to store helm charts. Harbor is multi-tenant : Project-specific configuration is possible with specific quota and policies. It also has the most complete set of features of any container registry we have ever worked with, such as: Connection with OpenID Connect. Audit logging for container pull and push actions. Policies for container images like tag retention and tag immutability. Replication to other registries (for example ECR, ACR, ...). Webhooks can be triggered when specific actions occur. A project can serve as a cache proxy for public images. Additionally, Harbor has a well-documented API, and we use a Terraform provider to set up resources like projects and users within Harbor using Terraform code. As you can read, Harbor has a lot to offer. ;-) In the next section, we take a deeper look into some of the features we haven’t covered yet, but are definitely worth mentioning. Container Image Vulnerability Scanning One of the most interesting features is the built-in Vulnerability Scanning. Harbor has a built-in vulnerability scanning tool that automatically scans container images for vulnerabilities when pushed to the registry. A job will be scheduled when a container image is pushed and once the result is available, it will be visible next to the container image. You can also get more details on the specific vulnerabilities that are found: For us, having these insights on the quality of the containers is a huge improvement compared to the current container registry we are using. By making some additional configurations, we can make the vulnerability scanning experience even better. Some examples: Block pulling container images that have Critical CVE vulnerabilities. Shedule frequent scans of images already stored in the Harbor registery. Allow specific CVEs that can't be fixed at the moment. Set up a webhook to take action when a CVE is detected. Harbor uses Trivy as its default vulnerability scanning tool, but it's easy to switch to another tool by installing it on your cluster and registering it in the Harbor interface. With these simple steps, you can take container security to the next level. Container Image Signing Harbor also offers a Container Image Signing feature that allows users to verify the trust of container images. When Notary or Cosign is used to secure container images, Harbor can validate their signatures, ensuring that the images have not been tampered with by any unauthorized sources other than your build tools. The Container Image Signing feature is signified by a green check mark in Harbor's interface, indicating that the image has been correctly signed. While this blog post won't cover how this feature works, you can find detailed documentation on the process via this link . Robot Accounts In addition to regular users, Harbor also allows for the creation of robot accounts. These system users are not associated with personal accounts and are often utilized by scripts and processes to authenticate with the Harbor registry. For instance, when building a container image, scripts may use a robot account to push the container image to the Harbor registry. To increase security, it's possible to set up an expiration time for robot users. Moreover, the access rights can be restricted to a particular project, and even the level of permissions within that project can be customized. The audit log records all activities performed by the robot account, just like regular users. What does the Harbor registery setup look like? We run Harbor registry on EKS, the Kubernetes Service provided by AWS. Since we are running on AWS, we can use some of the AWS services to provide some of the dependencies. We have 3 layers of configuration in this setup: AWS resources. Kubernetes resources. Resources within Harbor registery. ⬇️ In the next sections, we will take a deeper dive into these layers of configuration. 1. Setting up the AWS resources Within the ACA Group, we try to manage all our infrastructure as code. We use Terraform to setup all the dependencies for our Harbor registry: Route53 to provide DNS. RDS to provide a multi-az postgres database. Elasticache to provide a high-available Redis cluster for session management. EFS to provide multi-zone shared storage for our containers. EKS to provide the Kubernetes cluster master layer. Nodegroups deployed over multiple availability zones that will serve as compute capacity for our Kubernetes cluster. Once these resources are available, we can generate the Kubernetes configuration and deploy these to our Kubernetes cluster. 2. Setting up the kubernetes resources We use a helm chart to generate the Kubernetes configuration files that are required to set up Harbor. These are YAML files that are stored in GIT repositories. Ultimately, flux will deploy these YAML files to the Kubernetes cluster. ℹ️ you can read about deployments with flux in another blogpost here . As a result, the following workloads are created on your Kubernetes cluster: Harbor core. Harbor Portal, serves the UI. Harbor Registry, manages the container registry. Harbor Jobservice, schedules background jobs. Harbor Trivy, CVE / vulnerability scanning. Notary resources, image signing. Additionally, various Kubernetes objects are generated, including the Ingress, which exposes the user interface (UI) on your specified URL. If you want more details, you can directly install Harbor on your local Kubernetes cluster by running the Helm install command: helm install my-release harbor/harbor Default 3. Seting up resources within Harbor Now that we have the Harbor registry up and running, we can efficiently create various resources such as projects, robot accounts, and retention policies within it. Once again, we want to manage these resources in code instead of creating them via the UI. This not only helps us track the changes effectively but also prevents any potential misconfigurations through pull request mechanisms. With its comprehensive and well-documented API, Harbor allows many tools to develop custom addons. Leveraging our expertise in Terraform code, we prefer the Terraform Harbor addon to efficiently manage the resources within the Harbor registry. The following example will create a project within Harbor: resource "harbor_project" "myproject" { name = "myproject" public = false vulnerability_scanning = true enable_content_trust = true deployment_security = "" } Default Using the Harbor registery After deploying the Harbor registry and creating projects, it becomes a functional container registry that operates similarly to any other container registry. To push container images to the Harbor registry, conventional build jobs can be used. However, the build job requires authentication credentials, usually from a robot account. Following that, you have to update the configuration of your jenkins, tekton, BitBucket Pipeline, GitHub action, or similar job to specify the correct project and the Harbor URL, such as registry.example.be/myproject. The push command can also be found in the Harbor interface: After pushing the container image to the Harbor registry, it can be pulled from other locations. To pull an image to your local machine using Docker, you can use the following commands: docker login docker pull registry.example.be/myproject/image:version Default To utilize the container image in a Kubernetes environment, start by creating a Secret with the "docker-registry" type that includes the necessary credentials for deploying the container image. As a Secret is specific to a namespace, you need to run this command for each namespace that uses a container from the Harbor registry. kubectl -n NAMESPACE create secret docker-registry registry.example.be --docker-server=registry.example.be --docker-username='firstname.lastname' --docker-password='mysupersecurepassword' --docker-email=me@company.be Default Now you can point to the container image within your Deployment, StatefulSet, Job, … The imagePullSecrets section points to the Secret created in the step above. image: registry.example.be/myproject/image:version … imagePullSecrets: - name: registry.example.be Default Conclusion This blog post provided an overview of the numerous advantages and features of the Harbor registry. We also shared our approach to setting up and utilizing the container registry. At ACA, we use Harbor as the container registry for one of our most important projects and are currently in the process of adopting it as our default registry for new projects. Once it has been set up, we will create a plan to migrate additional active projects. Our goal is to enhance stability, availability and security for our clients. If you would like to learn more about Harbor registry, feel free to contact us! {% module_block module "widget_32015f12-8114-463e-bcf8-473d84a7e2dd" %}{% module_attribute "buttons" is_json="true" %}{% raw %}[{"appearance":{"link_color":"light","primary_color":"primary","secondary_color":"primary","tertiary_color":"light","tertiary_icon_accent_color":"dark","tertiary_text_color":"dark","variant":"primary"},"content":{"arrow":"right","icon":{"alt":null,"height":null,"loading":"disabled","size_type":null,"src":"","width":null},"tertiary_icon":{"alt":null,"height":null,"loading":"disabled","size_type":null,"src":"","width":null},"text":"Talk to use here!"},"target":{"link":{"no_follow":false,"open_in_new_tab":false,"rel":"","sponsored":false,"url":{"content_id":null,"href":"https://www.acagroup.be/en/services/cloud/","href_with_scheme":"https://www.acagroup.be/en/services/cloud/","type":"EXTERNAL"},"user_generated_content":false}},"type":"normal"}]{% endraw %}{% end_module_attribute %}{% module_attribute "child_css" is_json="true" %}{% raw %}{}{% endraw %}{% end_module_attribute %}{% module_attribute "css" is_json="true" %}{% raw %}{}{% endraw %}{% end_module_attribute %}{% module_attribute "definition_id" is_json="true" %}{% raw %}null{% endraw %}{% end_module_attribute %}{% module_attribute "field_types" is_json="true" %}{% raw %}{"buttons":"group","styles":"group"}{% endraw %}{% end_module_attribute %}{% module_attribute "isJsModule" is_json="true" %}{% raw %}true{% endraw %}{% end_module_attribute %}{% module_attribute "label" is_json="true" %}{% raw %}null{% endraw %}{% end_module_attribute %}{% module_attribute "module_id" is_json="true" %}{% raw %}201493994716{% endraw %}{% end_module_attribute %}{% module_attribute "path" is_json="true" %}{% raw %}"@projects/aca-group-project/aca-group-app/components/modules/ButtonGroup"{% endraw %}{% end_module_attribute %}{% module_attribute "schema_version" is_json="true" %}{% raw %}2{% endraw %}{% end_module_attribute %}{% module_attribute "smart_objects" is_json="true" %}{% raw %}null{% endraw %}{% end_module_attribute %}{% module_attribute "smart_type" is_json="true" %}{% raw %}"NOT_SMART"{% endraw %}{% end_module_attribute %}{% module_attribute "tag" is_json="true" %}{% raw %}"module"{% endraw %}{% end_module_attribute %}{% module_attribute "type" is_json="true" %}{% raw %}"module"{% endraw %}{% end_module_attribute %}{% module_attribute "wrap_field_tag" is_json="true" %}{% raw %}"div"{% endraw %}{% end_module_attribute %}{% end_module_block %}