{"id":5192,"date":"2025-08-13T17:36:01","date_gmt":"2025-08-13T17:36:01","guid":{"rendered":"https:\/\/b2f.pt\/?p=5192"},"modified":"2026-10-15T11:32:29","modified_gmt":"2026-10-15T11:32:29","slug":"guia-de-otimizacao-spark-no-microsoft-fabric","status":"publish","type":"post","link":"https:\/\/b2f.pt\/en\/guia-de-otimizacao-spark-no-microsoft-fabric\/","title":{"rendered":"Spark Optimization Guide in Microsoft Fabric"},"content":{"rendered":"<p>In the universe of <strong>Business Intelligence<\/strong> moderno, lidar com grandes volumes de dados de forma eficiente \u00e9 um desafio constante. Neste contexto, o <strong>Microsoft Fabric<\/strong> surge como uma plataforma unificada que simplifica a arquitetura de dados.<\/p>\n\n\n\n<p>Al\u00e9m disso, a sua integra\u00e7\u00e3o com o <strong>Apache Spark<\/strong> is at the heart of its large-scale data processing capability.<\/p>\n\n\n\n<p>Neste artigo, vamos explorar como esta poderosa combina\u00e7\u00e3o funciona. Para isso, come\u00e7aremos por compreender o papel do Spark no <strong>processamento distribu\u00eddo<\/strong> e as suas principais vantagens.<\/p>\n\n\n\n<p>Em seguida, mostraremos como configurar e melhorar a <strong>otimiza\u00e7\u00e3o Spark no Microsoft Fabric<\/strong>, atrav\u00e9s de diferentes configura\u00e7\u00f5es de clusters.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. Overview of Microsoft Fabric<\/h2>\n\n\n\n<p>The <strong>Microsoft Fabric<\/strong> consolida diferentes funcionalidades numa \u00fanica plataforma, facilitando a gest\u00e3o integrada da informa\u00e7\u00e3o.<\/p>\n\n\n\n<p>Entre estas funcionalidades, destacam-se <strong>Data Warehousing<\/strong>, <strong>Data Engineering<\/strong>, processamento em tempo real, <strong>Data Science<\/strong> and <strong>Machine Learning<\/strong>.<\/p>\n\n\n\n<p>Desta forma, as organiza\u00e7\u00f5es conseguem gerir diferentes processos anal\u00edticos num ambiente unificado.<\/p>\n\n\n<div class=\"wp-block-image is-style-rounded\">\n<figure class=\"aligncenter size-full is-resized\"><a href=\"https:\/\/b2f.pt\/en\/business-intelligence-analytics\/\" target=\"_blank\" rel=\" noreferrer noopener\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"428\" src=\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/melhorar-performance-spark-microsoft-fabric.png\" alt=\"Configura\u00e7\u00e3o e otimiza\u00e7\u00e3o de processos Apache Spark para melhorar o desempenho no Microsoft Fabric.\" class=\"wp-image-5843\" style=\"width:443px;height:auto\" srcset=\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/melhorar-performance-spark-microsoft-fabric.png 1024w, https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/melhorar-performance-spark-microsoft-fabric-300x125.png 300w, https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/melhorar-performance-spark-microsoft-fabric-768x321.png 768w, https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/melhorar-performance-spark-microsoft-fabric-18x8.png 18w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><\/figure><\/div>\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">2. How Does Spark Work?<\/h2>\n\n\n\n<p>The <strong>Apache Spark<\/strong> \u00e9 um mecanismo de <strong>processamento distribu\u00eddo<\/strong> que funciona atrav\u00e9s de uma arquitetura <em>master-worker<\/em>, com paralelismo inerente.<\/p>\n\n\n\n<p>Assim, permite processar grandes volumes de dados atrav\u00e9s de m\u00faltiplas m\u00e1quinas de forma coordenada.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2.1. Parallelism Architecture in Spark<\/h3>\n\n\n\n<p>Spark operates on a hierarchical architecture composed of two main types of nodes: the <strong>master node<\/strong> (also referred to as <strong>driver<\/strong>) and the <strong>worker nodes<\/strong>, que executam os <strong>executors<\/strong>.<\/p>\n\n\n\n<p>Esta distribui\u00e7\u00e3o permite ao Spark dividir tarefas complexas em opera\u00e7\u00f5es mais pequenas. Desta forma, diferentes componentes conseguem executar essas opera\u00e7\u00f5es em paralelo.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Driver Node<\/h4>\n\n\n\n<p>The <strong>driver node<\/strong> atua como o coordenador central do cluster.<\/p>\n\n\n\n<p>Neste sentido, assume as seguintes responsabilidades:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Analisar, distribuir e agendar tarefas entre os executors.<\/li>\n\n\n\n<li>Maintaining the <strong>SparkContext<\/strong>, que representa a liga\u00e7\u00e3o ao cluster Spark.<\/li>\n\n\n\n<li>Monitorizar o progresso da execu\u00e7\u00e3o e garantir toler\u00e2ncia a falhas.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Worker Nodes e Executors<\/h4>\n\n\n\n<p>Por outro lado, os <strong>worker nodes<\/strong> contain the <strong>executors<\/strong>, que s\u00e3o processos respons\u00e1veis pela execu\u00e7\u00e3o real das tarefas.<\/p>\n\n\n\n<p>Cada executor tem duas responsabilidades principais:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Executar o c\u00f3digo atribu\u00eddo pelo driver\/worker node.<\/li>\n\n\n\n<li>Reportar o progresso dos c\u00e1lculos de volta ao driver node.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2.2. How Parallelism Works<\/h3>\n\n\n\n<p>O Spark alcan\u00e7a o paralelismo atrav\u00e9s da divis\u00e3o dos dados em <strong>partitions<\/strong>, que distribui pelos diferentes n\u00f3s do cluster.<\/p>\n\n\n\n<p>Desta forma, diferentes <em>threads<\/em> processam cada parti\u00e7\u00e3o de forma <strong>independente<\/strong>, permitindo executar opera\u00e7\u00f5es simult\u00e2neas.<\/p>\n\n\n\n<p>Por exemplo, se um <em>dataset<\/em> estiver dividido em m\u00faltiplas parti\u00e7\u00f5es de 128 MB, diferentes executors podem process\u00e1-las em paralelo.<\/p>\n\n\n\n<p>Consequentemente, esta distribui\u00e7\u00e3o permite maximizar a utiliza\u00e7\u00e3o dos recursos computacionais.<\/p>\n\n\n\n<p>Al\u00e9m disso, o Spark cria um <strong>DAG (Directed Acyclic Graph)<\/strong> para programar tarefas e orquestrar os n\u00f3s <em>worker<\/em> no cluster.<\/p>\n\n\n\n<p>Este mecanismo permite otimizar a sequ\u00eancia de opera\u00e7\u00f5es. Por outro lado, facilita a recupera\u00e7\u00e3o em caso de falhas, replicando apenas as opera\u00e7\u00f5es necess\u00e1rias nos dados de um estado anterior.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. Spark Configurations in Microsoft Fabric<\/h2>\n\n\n\n<p>A configura\u00e7\u00e3o dos clusters \u00e9 uma etapa importante na <strong>otimiza\u00e7\u00e3o Spark no Microsoft Fabric<\/strong>.<\/p>\n\n\n\n<p>Por isso, \u00e9 fundamental considerar os requisitos de acesso, o dimensionamento dos recursos e as caracter\u00edsticas dos processos.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3.1. Prerequisites<\/h3>\n\n\n\n<p>Antes de iniciar a configura\u00e7\u00e3o, \u00e9 necess\u00e1rio garantir:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Acesso ao portal do <strong>Microsoft Fabric<\/strong> com as permiss\u00f5es necess\u00e1rias (<em>admin<\/em>, <em>contributor<\/em> or <em>member<\/em>).<\/li>\n\n\n\n<li>SKU de Fabric previamente contratado ou <strong>Fabric Trial<\/strong> ativo.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">3.2. Configuration Steps<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Pre-warmed Cluster Configuration<\/h4>\n\n\n\n<p>The <strong>Microsoft Fabric<\/strong> oferece <strong>Starter Pools<\/strong> that use clusters <strong>pre-warmed<\/strong>, executados em m\u00e1quinas virtuais da Microsoft.<\/p>\n\n\n\n<p>Desta forma, conseguem <strong>reduzir significativamente os tempos de in\u00edcio<\/strong>.<\/p>\n\n\n\n<p>Estes clusters est\u00e3o ativos e preparados para utiliza\u00e7\u00e3o, proporcionando a inicializa\u00e7\u00e3o de sess\u00f5es Spark tipicamente entre 5 e 10 segundos.<\/p>\n\n\n\n<p>Al\u00e9m disso, n\u00e3o exigem configura\u00e7\u00e3o manual.<\/p>\n\n\n\n<p>Os Starter Pools utilizam n\u00f3s de tamanho m\u00e9dio, que escalam dinamicamente de acordo com as necessidades dos trabalhos Spark.<\/p>\n\n\n\n<p>Assim, quando n\u00e3o existem depend\u00eancias de bibliotecas personalizadas ou propriedades Spark customizadas, as sess\u00f5es podem iniciar quase instantaneamente.<\/p>\n\n\n\n<p>Isto acontece porque o cluster j\u00e1 se encontra em execu\u00e7\u00e3o e n\u00e3o necessita de tempo adicional de provisionamento.<\/p>\n\n\n\n<p>No entanto, existem cen\u00e1rios em que o tempo de in\u00edcio pode ser superior:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Bibliotecas personalizadas:<\/strong> Podem acrescentar entre 30 segundos e 5 minutos para personaliza\u00e7\u00e3o da sess\u00e3o.<\/li>\n\n\n\n<li><strong>Alta utiliza\u00e7\u00e3o regional:<\/strong> Quando os Starter Pools est\u00e3o saturados, a cria\u00e7\u00e3o de novos clusters pode demorar entre 2 e 5 minutos.<\/li>\n\n\n\n<li><strong>Op\u00e7\u00f5es de rede:<\/strong> Private Links ou Managed VNets podem desativar os Starter Pools, obrigando \u00e0 cria\u00e7\u00e3o de clusters <em>on-demand<\/em>.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">High Concurrency Mode Activation<\/h4>\n\n\n\n<p>It is recommended to enable the <strong>High Concurrency Mode<\/strong> para permitir que m\u00faltiplos notebooks partilhem a mesma sess\u00e3o Spark.<\/p>\n\n\n\n<p>Desta forma, \u00e9 poss\u00edvel otimizar a utiliza\u00e7\u00e3o de recursos e reduzir significativamente os tempos de in\u00edcio.<\/p>\n\n\n\n<p>Al\u00e9m disso, os pools personalizados com <em>High Concurrency<\/em> proporcionam uma experi\u00eancia de in\u00edcio de sess\u00e3o mais r\u00e1pida, comparativamente \u00e0s sess\u00f5es Spark standard.<\/p>\n\n\n\n<p>To enable High Concurrency Mode:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Access the <strong>Workspace Settings<\/strong>.<\/li>\n\n\n\n<li>Navigate to <strong>Data Engineering\/Science > Spark Settings > High Concurrency<\/strong>.<\/li>\n\n\n\n<li>Enable the option <strong>For notebooks<\/strong>.<\/li>\n<\/ol>\n\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<div class=\"wp-block-image is-style-default\">\n<figure class=\"aligncenter size-full is-resized\"><a href=\"https:\/\/b2f.pt\/en\/business-intelligence-analytics\/\" target=\"_blank\" rel=\" noreferrer noopener\"><img decoding=\"async\" width=\"635\" height=\"263\" src=\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/processamento-dados-apache-spark-fabric.png\" alt=\"Processamento distribu\u00eddo e an\u00e1lise de grandes volumes de dados com Apache Spark no Microsoft Fabric.\" class=\"wp-image-5844\" style=\"width:592px;height:auto\" srcset=\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/processamento-dados-apache-spark-fabric.png 635w, https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/processamento-dados-apache-spark-fabric-300x124.png 300w, https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/processamento-dados-apache-spark-fabric-18x7.png 18w\" sizes=\"(max-width: 635px) 100vw, 635px\" \/><\/a><\/figure><\/div>\n\n\n<div style=\"height:30px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h4 class=\"wp-block-heading\">EEspecifica\u00e7\u00f5es Recomendadas para Pools<\/h4>\n\n\n\n<p>Para demonstrar as configura\u00e7\u00f5es ideais, consideremos um exemplo com o SKU <strong>F64<\/strong>.<\/p>\n\n\n\n<p><strong>Base Capacity:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>F64 = 64 Capacity Units = 128 Spark VCores.<\/li>\n\n\n\n<li>Com um <strong>burst factor 3x<\/strong>, \u00e9 poss\u00edvel atingir um m\u00e1ximo de 384 Spark VCores.<\/li>\n<\/ul>\n\n\n\n<p>Neste caso, o <em>burst factor<\/em> multiplica a capacidade de processamento dispon\u00edvel, permitindo aumentar o desempenho.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th><strong>Parameter<\/strong><\/th><th><strong>Valor Recomendado<\/strong><\/th><th><strong>Explanation<\/strong><\/th><\/tr><\/thead><tbody><tr><td><strong>Node Family<\/strong><\/td><td>Memory Optimized<\/td><td>Suitable for data processing workloads<\/td><\/tr><tr><td><strong>Node Size<\/strong><\/td><td>Medium (8 VCores)<\/td><td>Balance between performance and concurrency<\/td><\/tr><tr><td><strong>Autoscale<\/strong><\/td><td>Enabled (min: 2, max: 48)<\/td><td>48 nodes \u00d7 8 VCores = 384 VCores (maximum burst)<\/td><\/tr><tr><td><strong>Dynamic Allocation<\/strong><\/td><td>Enabled<\/td><td>Allows automatic adjustment of executors based on demand<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p><strong>Dynamic Allocation Configuration:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Min Executors:<\/strong> 2 (baseline para disponibilidade imediata).<\/li>\n\n\n\n<li><strong>Max Executors:<\/strong> 46 (reservando 2 n\u00f3s para driver e overhead).<\/li>\n\n\n\n<li><strong>Initial Executors:<\/strong> 4 (equil\u00edbrio entre tempo de arranque e utiliza\u00e7\u00e3o de recursos).<\/li>\n<\/ul>\n\n\n\n<p><strong>Scaling Based on SKU<\/strong>:<\/p>\n\n\n\n<p>For different SKUs, the maximum configurations vary:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th><strong>SKU<\/strong><\/th><th><strong>Capacity Units<\/strong><\/th><th><strong>Max Spark VCores<\/strong><\/th><th><strong>Recommended Node Size<\/strong><\/th><th><strong>Max Nodes<\/strong><\/th><\/tr><\/thead><tbody><tr><td>F2<\/td><td>2<\/td><td>12<\/td><td>Small<\/td><td>3<\/td><\/tr><tr><td>F8<\/td><td>8<\/td><td>48<\/td><td>Medium<\/td><td>6<\/td><\/tr><tr><td>F16<\/td><td>16<\/td><td>96<\/td><td>Medium<\/td><td>12<\/td><\/tr><tr><td>F64<\/td><td>64<\/td><td>384<\/td><td>Medium\/Large<\/td><td>48\/24<\/td><\/tr><tr><td>F128<\/td><td>128<\/td><td>768<\/td><td>Large<\/td><td>48<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<div style=\"height:29px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Dynamic Allocation Configuration:<\/strong><\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Min Executors:<\/strong> 2 (baseline para disponibilidade imediata).<\/li>\n\n\n\n<li><strong>Max Executors:<\/strong> 46 (reservando 2 n\u00f3s para driver e overhead).<\/li>\n\n\n\n<li><strong>Initial Executors:<\/strong> 4 (equil\u00edbrio entre tempo de arranque e utiliza\u00e7\u00e3o de recursos).<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Scaling Based on SKU<\/h4>\n\n\n\n<p>Para diferentes SKUs, as configura\u00e7\u00f5es m\u00e1ximas variam.<\/p>\n\n\n\n<p>Por isso, \u00e9 importante avaliar os recursos dispon\u00edveis antes de definir o n\u00famero e o tamanho dos n\u00f3s.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th><strong>SKU<\/strong><\/th><th><strong>Capacity Units<\/strong><\/th><th><strong>Max Spark VCores<\/strong><\/th><th><strong>Recommended Node Size<\/strong><\/th><th><strong>Max Nodes<\/strong><\/th><\/tr><tr><td>F2<\/td><td>2<\/td><td>12<\/td><td>Small<\/td><td>3<\/td><\/tr><tr><td>F8<\/td><td>8<\/td><td>48<\/td><td>Medium<\/td><td>6<\/td><\/tr><tr><td>F16<\/td><td>16<\/td><td>96<\/td><td>Medium<\/td><td>12<\/td><\/tr><tr><td>F64<\/td><td>64<\/td><td>384<\/td><td>Medium\/Large<\/td><td>48\/24<\/td><\/tr><tr><td>F128<\/td><td>128<\/td><td>768<\/td><td>Large<\/td><td>48<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">Configuration Through the Portal<\/h4>\n\n\n\n<p>Depois de definir os requisitos do cluster, \u00e9 poss\u00edvel avan\u00e7ar para a configura\u00e7\u00e3o no portal do <strong>Microsoft Fabric<\/strong>.<\/p>\n\n\n\n<p><strong>In the Microsoft Fabric portal:<\/strong><\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Go to the section <strong>Engenharia de Dados > Spark Settings<\/strong>.<\/li>\n\n\n\n<li>Select <strong>New Pool<\/strong> para criar um cluster personalizado.<\/li>\n\n\n\n<li>Set the <strong>Node Family<\/strong> and <strong>Node Size<\/strong>, conforme os requisitos.<\/li>\n\n\n\n<li>Configurar o <strong>Autoscale<\/strong>, com n\u00famero m\u00ednimo de n\u00f3s igual a 1 (o Fabric garante disponibilidade restaur\u00e1vel mesmo com um \u00fanico n\u00f3).<\/li>\n\n\n\n<li>Enable <strong>Dynamic Executor Allocation<\/strong> para otimizar automaticamente os recursos.<\/li>\n<\/ol>\n\n\n\n<p><strong>Integration with Data Sources:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Utilizar conectores nativos para estabelecer liga\u00e7\u00f5es a <strong>Data Lakes<\/strong> or <strong>Data Warehouses<\/strong>.<\/li>\n\n\n\n<li>Al\u00e9m disso, verificar se as credenciais e configura\u00e7\u00f5es de seguran\u00e7a est\u00e3o corretamente definidas.<\/li>\n<\/ul>\n\n\n\n<p><strong>Notebook and Task Configuration:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Configurar notebooks (Python ou Scala) para desenvolver scripts de transforma\u00e7\u00e3o.<\/li>\n\n\n\n<li>Posteriormente, agendar tarefas <em>batch<\/em> ou configurar processos <em>streaming<\/em>, conforme os requisitos.<\/li>\n<\/ul>\n\n\n\n<p>Custom pools have a <strong>autopause<\/strong> padr\u00e3o de 2 minutos.<\/p>\n\n\n\n<p>Ap\u00f3s esse per\u00edodo de inatividade, as sess\u00f5es expiram e os clusters deixam de ter recursos alocados. Desta forma, a plataforma cobra apenas pelo tempo de utiliza\u00e7\u00e3o ativa.<\/p>\n\n\n\n<p>No entanto, as parametriza\u00e7\u00f5es ideais do cluster podem variar consoante os processos que ser\u00e3o executados.<\/p>\n\n\n\n<p>Por isso, \u00e9 fundamental realizar uma avalia\u00e7\u00e3o acompanhada de testes para cada caso.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Best Practices and Technical Considerations<\/h2>\n\n\n\n<p>A <strong>otimiza\u00e7\u00e3o Spark no Microsoft Fabric<\/strong> exige uma avalia\u00e7\u00e3o adequada dos recursos dispon\u00edveis e das caracter\u00edsticas de cada processo.<\/p>\n\n\n\n<p>Neste sentido, \u00e9 importante considerar o dimensionamento, a automatiza\u00e7\u00e3o, a monitoriza\u00e7\u00e3o e a seguran\u00e7a.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Sizing and Optimization<\/h4>\n\n\n\n<p>\u00c9 fundamental dimensionar corretamente o cluster.<\/p>\n\n\n\n<p>Para isso, considere os seguintes fatores:<\/p>\n\n\n\n<p><strong>Burst Factor:<\/strong> Determine a capacidade de escalabilidade instant\u00e2nea necess\u00e1ria para lidar com picos de processamento.<\/p>\n\n\n\n<p>Neste caso, a l\u00f3gica deve incluir a multiplica\u00e7\u00e3o do n\u00famero de VCores utilizados na sele\u00e7\u00e3o. Assim, \u00e9 poss\u00edvel atingir o <em>burst factor<\/em> pretendido.<\/p>\n\n\n\n<p>Por exemplo, para um SKU F64, com 128 VCores base, configure o pool para atingir at\u00e9 384 VCores.<\/p>\n\n\n\n<p>Para isso, ajuste o n\u00famero e o tamanho dos n\u00f3s, como no exemplo: Medium nodes (8 VCores cada) \u00d7 48 n\u00f3s = 384 VCores.<\/p>\n\n\n\n<p><strong>N\u00famero de Cores e Mem\u00f3ria:<\/strong> Para o <strong>driver node<\/strong>, selecione um n\u00famero adequado de cores e mem\u00f3ria.<\/p>\n\n\n\n<p>Isto \u00e9 importante porque este n\u00f3 coordena o processamento e deve suportar as tarefas de gest\u00e3o do cluster.<\/p>\n\n\n\n<p>Por outro lado, nos <strong>worker nodes<\/strong>, a escolha deve considerar a multiplica\u00e7\u00e3o dos recursos necess\u00e1rios para o processamento paralelo, nomeadamente n\u00facleos e mem\u00f3ria.<\/p>\n\n\n\n<p>Al\u00e9m disso, avalie a possibilidade de escalar estes n\u00f3s para ajustar o desempenho conforme a carga.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Automation and Scheduling<\/h4>\n\n\n\n<p>Automatize processos recorrentes atrav\u00e9s de scripts e agendamentos.<\/p>\n\n\n\n<p>Desta forma, consegue garantir maior consist\u00eancia e minimizar erros.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Monitoring<\/h4>\n\n\n\n<p>Utilize as ferramentas de <strong>monitoriza\u00e7\u00e3o nativas do Microsoft Fabric<\/strong> para identificar poss\u00edveis problemas.<\/p>\n\n\n\n<p>Assim, pode ajustar o desempenho do cluster em tempo real.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Security<\/h4>\n\n\n\n<p>Assegure a implementa\u00e7\u00e3o de <strong>pol\u00edticas de seguran\u00e7a robustas<\/strong>, configurando as permiss\u00f5es necess\u00e1rias.<\/p>\n\n\n\n<p>Al\u00e9m disso, utilize liga\u00e7\u00f5es seguras para garantir o acesso adequado aos dados.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Quais os Pr\u00f3ximos Passos?<\/h2>\n\n\n\n<p>A <strong><a href=\"https:\/\/b2f.pt\/en\/contacts\/\" target=\"_blank\" rel=\"noreferrer noopener\">B2F <\/a><\/strong>disp\u00f5e de uma s\u00f3lida experi\u00eancia no desenvolvimento de solu\u00e7\u00f5es em <strong>Microsoft Fabric<\/strong>, assim como na implementa\u00e7\u00e3o de processos baseados em <strong>Apache Spark<\/strong>.<\/p>\n\n\n\n<p>Caso necessite de apoio especializado para maximizar o desempenho e a efici\u00eancia da sua plataforma anal\u00edtica, a nossa equipa est\u00e1 preparada para colaborar consigo.<\/p>\n\n\n\n<p>Desta forma, podemos encontrar as solu\u00e7\u00f5es mais adequadas aos desafios espec\u00edficos da sua organiza\u00e7\u00e3o.<\/p>","protected":false},"excerpt":{"rendered":"<p>No universo do Business Intelligence moderno, lidar com grandes volumes de dados de forma eficiente \u00e9 um desafio constante. Neste contexto, o Microsoft Fabric surge como uma plataforma unificada que simplifica a arquitetura de dados. Al\u00e9m disso, a sua integra\u00e7\u00e3o com o Apache Spark \u00e9 o cora\u00e7\u00e3o da sua capacidade de processamento de dados em [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":5207,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[79],"class_list":["post-5192","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-business-intelligence","tag-spark"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\r\n<title>Otimiza\u00e7\u00e3o Spark no Microsoft Fabric: Guia Completo Copiar<\/title>\r\n<meta name=\"description\" content=\"Descubra como melhorar a otimiza\u00e7\u00e3o Spark no Microsoft Fabric, configurar clusters e aumentar o desempenho do processamento de dados.\" \/>\r\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\r\n<link rel=\"canonical\" href=\"https:\/\/b2f.pt\/en\/guia-de-otimizacao-spark-no-microsoft-fabric\/\" \/>\r\n<meta property=\"og:locale\" content=\"en_GB\" \/>\r\n<meta property=\"og:type\" content=\"article\" \/>\r\n<meta property=\"og:title\" content=\"Otimiza\u00e7\u00e3o Spark no Microsoft Fabric: Guia Completo Copiar\" \/>\r\n<meta property=\"og:description\" content=\"Descubra como melhorar a otimiza\u00e7\u00e3o Spark no Microsoft Fabric, configurar clusters e aumentar o desempenho do processamento de dados.\" \/>\r\n<meta property=\"og:url\" content=\"https:\/\/b2f.pt\/en\/guia-de-otimizacao-spark-no-microsoft-fabric\/\" \/>\r\n<meta property=\"og:site_name\" content=\"B2F - Business to Future\" \/>\r\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/businesstofuture\/?locale=pt_PT\" \/>\r\n<meta property=\"article:published_time\" content=\"2025-08-13T17:36:01+00:00\" \/>\r\n<meta property=\"article:modified_time\" content=\"2026-10-15T11:32:29+00:00\" \/>\r\n<meta property=\"og:image\" content=\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg\" \/>\r\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\r\n\t<meta property=\"og:image:height\" content=\"2398\" \/>\r\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\r\n<meta name=\"author\" content=\"b2f\" \/>\r\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\r\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"b2f\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\r\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/\"},\"author\":{\"name\":\"b2f\",\"@id\":\"https:\/\/b2f.pt\/#\/schema\/person\/53a60449eab118b3e0c1818995da0497\"},\"headline\":\"Guia de Otimiza\u00e7\u00e3o Spark no Microsoft Fabric\",\"datePublished\":\"2025-08-13T17:36:01+00:00\",\"dateModified\":\"2026-10-15T11:32:29+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/\"},\"wordCount\":1754,\"publisher\":{\"@id\":\"https:\/\/b2f.pt\/#organization\"},\"image\":{\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg\",\"keywords\":[\"Spark\"],\"articleSection\":[\"Business Intelligence\"],\"inLanguage\":\"en-GB\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/\",\"url\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/\",\"name\":\"Otimiza\u00e7\u00e3o Spark no Microsoft Fabric: Guia Completo Copiar\",\"isPartOf\":{\"@id\":\"https:\/\/b2f.pt\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg\",\"datePublished\":\"2025-08-13T17:36:01+00:00\",\"dateModified\":\"2026-10-15T11:32:29+00:00\",\"description\":\"Descubra como melhorar a otimiza\u00e7\u00e3o Spark no Microsoft Fabric, configurar clusters e aumentar o desempenho do processamento de dados.\",\"breadcrumb\":{\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage\",\"url\":\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg\",\"contentUrl\":\"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg\",\"width\":2560,\"height\":2398},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/b2f.pt\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Guia de Otimiza\u00e7\u00e3o Spark no Microsoft Fabric\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/b2f.pt\/#website\",\"url\":\"https:\/\/b2f.pt\/\",\"name\":\"B2F - Business to Future\",\"description\":\"Retire Conhecimento Dos Seus Dados\",\"publisher\":{\"@id\":\"https:\/\/b2f.pt\/#organization\"},\"alternateName\":\"B2F\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/b2f.pt\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/b2f.pt\/#organization\",\"name\":\"B2F - Business to Future\",\"alternateName\":\"B2F\",\"url\":\"https:\/\/b2f.pt\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/b2f.pt\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/b2f.pt\/wp-content\/uploads\/2026\/10\/b2f_horizontal_blue_h100px.png\",\"contentUrl\":\"https:\/\/b2f.pt\/wp-content\/uploads\/2026\/10\/b2f_horizontal_blue_h100px.png\",\"width\":276,\"height\":102,\"caption\":\"B2F - Business to Future\"},\"image\":{\"@id\":\"https:\/\/b2f.pt\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/businesstofuture\/?locale=pt_PT\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/b2f.pt\/#\/schema\/person\/53a60449eab118b3e0c1818995da0497\",\"name\":\"b2f\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/b2f.pt\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b44a9c116c175f8fb789cb42f4dcedc8051cd52bd25399a21d294fe1f984bbc9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b44a9c116c175f8fb789cb42f4dcedc8051cd52bd25399a21d294fe1f984bbc9?s=96&d=mm&r=g\",\"caption\":\"b2f\"},\"sameAs\":[\"https:\/\/b2f.pt\"],\"url\":\"https:\/\/b2f.pt\/en\/author\/b2f\/\"}]}<\/script>\r\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Otimiza\u00e7\u00e3o Spark no Microsoft Fabric: Guia Completo Copiar","description":"Descubra como melhorar a otimiza\u00e7\u00e3o Spark no Microsoft Fabric, configurar clusters e aumentar o desempenho do processamento de dados.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/b2f.pt\/en\/guia-de-otimizacao-spark-no-microsoft-fabric\/","og_locale":"en_GB","og_type":"article","og_title":"Otimiza\u00e7\u00e3o Spark no Microsoft Fabric: Guia Completo Copiar","og_description":"Descubra como melhorar a otimiza\u00e7\u00e3o Spark no Microsoft Fabric, configurar clusters e aumentar o desempenho do processamento de dados.","og_url":"https:\/\/b2f.pt\/en\/guia-de-otimizacao-spark-no-microsoft-fabric\/","og_site_name":"B2F - Business to Future","article_publisher":"https:\/\/www.facebook.com\/businesstofuture\/?locale=pt_PT","article_published_time":"2025-08-13T17:36:01+00:00","article_modified_time":"2026-10-15T11:32:29+00:00","og_image":[{"width":2560,"height":2398,"url":"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg","type":"image\/jpeg"}],"author":"b2f","twitter_card":"summary_large_image","twitter_misc":{"Written by":"b2f","Estimated reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#article","isPartOf":{"@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/"},"author":{"name":"b2f","@id":"https:\/\/b2f.pt\/#\/schema\/person\/53a60449eab118b3e0c1818995da0497"},"headline":"Guia de Otimiza\u00e7\u00e3o Spark no Microsoft Fabric","datePublished":"2025-08-13T17:36:01+00:00","dateModified":"2026-10-15T11:32:29+00:00","mainEntityOfPage":{"@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/"},"wordCount":1754,"publisher":{"@id":"https:\/\/b2f.pt\/#organization"},"image":{"@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage"},"thumbnailUrl":"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg","keywords":["Spark"],"articleSection":["Business Intelligence"],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/","url":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/","name":"Otimiza\u00e7\u00e3o Spark no Microsoft Fabric: Guia Completo Copiar","isPartOf":{"@id":"https:\/\/b2f.pt\/#website"},"primaryImageOfPage":{"@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage"},"image":{"@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage"},"thumbnailUrl":"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg","datePublished":"2025-08-13T17:36:01+00:00","dateModified":"2026-10-15T11:32:29+00:00","description":"Descubra como melhorar a otimiza\u00e7\u00e3o Spark no Microsoft Fabric, configurar clusters e aumentar o desempenho do processamento de dados.","breadcrumb":{"@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#primaryimage","url":"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg","contentUrl":"https:\/\/b2f.pt\/wp-content\/uploads\/2025\/08\/imagem_comprimida-scaled.jpg","width":2560,"height":2398},{"@type":"BreadcrumbList","@id":"https:\/\/b2f.pt\/guia-de-otimizacao-spark-no-microsoft-fabric\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/b2f.pt\/"},{"@type":"ListItem","position":2,"name":"Guia de Otimiza\u00e7\u00e3o Spark no Microsoft Fabric"}]},{"@type":"WebSite","@id":"https:\/\/b2f.pt\/#website","url":"https:\/\/b2f.pt\/","name":"B2F - Business to Future","description":"Retire Conhecimento Dos Seus Dados","publisher":{"@id":"https:\/\/b2f.pt\/#organization"},"alternateName":"B2F","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/b2f.pt\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/b2f.pt\/#organization","name":"B2F - Business to Future","alternateName":"B2F","url":"https:\/\/b2f.pt\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/b2f.pt\/#\/schema\/logo\/image\/","url":"https:\/\/b2f.pt\/wp-content\/uploads\/2026\/10\/b2f_horizontal_blue_h100px.png","contentUrl":"https:\/\/b2f.pt\/wp-content\/uploads\/2026\/10\/b2f_horizontal_blue_h100px.png","width":276,"height":102,"caption":"B2F - Business to Future"},"image":{"@id":"https:\/\/b2f.pt\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/businesstofuture\/?locale=pt_PT"]},{"@type":"Person","@id":"https:\/\/b2f.pt\/#\/schema\/person\/53a60449eab118b3e0c1818995da0497","name":"b2f","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/b2f.pt\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b44a9c116c175f8fb789cb42f4dcedc8051cd52bd25399a21d294fe1f984bbc9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b44a9c116c175f8fb789cb42f4dcedc8051cd52bd25399a21d294fe1f984bbc9?s=96&d=mm&r=g","caption":"b2f"},"sameAs":["https:\/\/b2f.pt"],"url":"https:\/\/b2f.pt\/en\/author\/b2f\/"}]}},"_links":{"self":[{"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/posts\/5192","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/comments?post=5192"}],"version-history":[{"count":1,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/posts\/5192\/revisions"}],"predecessor-version":[{"id":5845,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/posts\/5192\/revisions\/5845"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/media\/5207"}],"wp:attachment":[{"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/media?parent=5192"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/categories?post=5192"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/b2f.pt\/en\/wp-json\/wp\/v2\/tags?post=5192"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}