Recent Releases and Announcements
- Cumulative Update #8 for SQL Server 2017 RTM
- Cumulative Update #12 for SQL Server 2014 SP2
- Microsoft releases the latest update of Analytics Platform System
- Microsoft is pleased to announce that the Analytics Platform System (APS) appliance update 7 (AU7) is now generally available. APS is Microsoft’s scale-out Massively Parallel Processing (MPP) system based on SQL Server for data warehouse specific workloads on-premises.
- https://cloudblogs.microsoft.com/sqlserver/2018/06/04/microsoft-releases-the-latest-update-of-analytics-platform-system/
- SSMS 17.8 is now available
- In this release of SQL Server Management Studio, we have introduced UI and scripting support for the AUTOGROW_ALL_FILES database filegroup property.
- This property was introduced in SQL Server 2016 to replace trace flag 1117, but it was only settable via T-SQL script. Now you can set the property via a checkbox in the Database Properties -> Filegroups page.
- https://cloudblogs.microsoft.com/sqlserver/2018/06/21/ssms-17-8-is-now-available/
- The June release of SQL Operations Studio is now available
- The SQL Server Profiler for SQL Operations Studio Preview Extension is now available in the Extension Manager. This is the initial preview release for a new lightweight XEvent-based profiler.
- https://cloudblogs.microsoft.com/sqlserver/2018/06/20/the-june-release-of-sql-operations-studio-is-now-available/
- Column-Level Security is now supported in Azure SQL Data Warehouse
- With the introduction of CLS, you can adjust permissions to view sensitive data by limiting user access to specific columns in your tables without having to redesign your data warehouse.
- https://azure.microsoft.com/en-us/blog/column-level-security-is-now-supported-in-azure-sql-data-warehouse/
- Announcing Event based trigger integration with Azure Data Factory
- A lot of data integration scenarios requires data factory customers to trigger pipelines based on events.
- A typical event could be file landing or getting deleted in your azure storage. As soon as the file arrives in your storage location and the corresponding blob is created, it can trigger and run your data factory pipeline.
- https://azure.microsoft.com/en-us/blog/event-trigger-based-scheduling-in-azure-data-factory/
- Resumable Online Index Create is in public preview for Azure SQL DB
- The feature allows the ability to pause an index create operation and resume it later from where the index create operation was paused or failed, rather than having to restart the operation from the beginning.
- https://azure.microsoft.com/en-us/blog/resumable-online-index-create-is-in-public-preview-for-azure-sql-db/
- Announcing the general availability of Azure SQL Data Sync
- Azure SQL Data Sync allows you to synchronize data between Azure SQL Database and any other SQL endpoints unidirectionally or bidirectionally. It enables hybrid SQL deployment and allows local data access from both Azure and on-premises application.
- https://azure.microsoft.com/en-us/blog/announcing-the-general-availability-of-azure-sql-data-sync/
- Announcing Azure Data Lake Storage Gen2
- We announced the preview of Azure Data Lake Storage Gen2 the only data lake designed specifically for enterprises to run large scale analytics workloads in the cloud.
- Azure Data Lake Storage Gen2 takes core capabilities from Azure Data Lake Storage Gen1 such as a Hadoop compatible file system, Azure Active Directory and POSIX based ACLs and integrates them into Azure Blob Storage.
- https://azure.microsoft.com/en-us/blog/a-closer-look-at-azure-data-lake-storage-gen2/
- Azure Backup for SQL Server on Azure now in public preview
- This is a new Azure Backup capability to backup SQL workloads running in Azure Virtual Machines
- https://azure.microsoft.com/en-us/blog/azure-backup-for-sql-server-on-azure-vm-public-preview/
- Power BI Service and Mobile May Feature Summary
- Power BI Desktop June Feature Summary
- DSC Resource Kit Release June 2018
Troubleshooting and Issue Alerts
- Critical: Do NOT delete files from the Windows Installer folder. C:windowsInstaller is not a temporary folder and files in it should not be deleted. If you do it on machines on which you have SQL Server installed, you may have to rebuild the operating system and reinstall SQL Server.
- Critical: Please be aware of a critical Microsoft Visual C++ 2013 runtime pre-requisite update that may be required on machines where SQL Server 2016 will be, or has been, installed.
- https://blogs.msdn.microsoft.com/sqlcat/2016/07/28/installing-sql-server-2016-rtm-you-must-do-this/
- If KB3164398 or KB3138367 are installed, then no further action is necessary. To check, run the following from a command prompt:
- powershell get-hotfix KB3164398
- powershell get-hotfix KB3138367
- If the version of %SystemRoot%system32msvcr120.dll is 12.0.40649.5 or later, then no further action is necessary. To check, run the following from a command prompt:
- powershell "get-item %systemroot%system32msvcr120.dll | select versioninfo | fl"
- Important: If the Update Cache folder or some patches are removed from this folder, you can no longer uninstall an update to your SQL Server instance and then revert to an earlier update build.
- In that situation, Add/Remove Programs entries point to non-existing binaries, and therefore the uninstall process does not work. Therefore, Microsoft strongly encourages you to keep the folder and its contents intact.
- https://support.microsoft.com/en-us/kb/3196535
- Important: Default auto statistics update threshold change for SQL Server 2016
- https://blogs.msdn.microsoft.com/psssql/2016/10/04/default-auto-statistics-update-threshold-change-for-sql-server-2016/
- SQL Server 2016: Default is new threshold if database compatibility level is 130. If database compatibility is below 130, old threshold is used (unless you use trace flag 2371)
- Optimizing Replication Agent profile parameters for better performance
- Log Reader Agent - ReadBatchSize: From the testing, it is evident that there is no one correct value that can be applicable in the profile to improve throughput of the log reader agent. At a high level, for workloads with smaller transactions, a higher value of ReadBatchSize may benefit throughput. However, for workloads with large transactions, changing the ReadBatchSize will not improve performance.
- Distribution Agent - CommitBatchSize and SubscriptionStreams: From the testing, it is evident that increasing the number of parallel streams and CommitBatchSize have a significant impact on the overall throughput of the system.
- https://blogs.msdn.microsoft.com/sql_server_team/optimizing-replication-agent-profile-parameters-for-better-performance/
- Replication error: The insert failed. It conflicted with an identity range check constraint
- Customer had set @identity_range to 10K and threshold was set to 80. We increased @identity_range to 100K. This increased the threshold range and prevented errors with the Insert.
- https://blogs.msdn.microsoft.com/repltalk/2018/06/02/the-insert-failed-it-conflicted-with-an-identity-range-check-constraint/
- SQL Server 2016/2017: Availability group secondary replica redo model and performance
Recent Blog Posts and Articles
- Detecting unconscious bias in models, with R
- For a practical demonstration of one way you can go about detecting proxy bias in R, take a look at the vignette created by Paige Bailey for the ROpenSci conference. The vignette details general principles you can follow to identify proxy bias in an analysis, in the context of a case study analyzed using R.
- The case study demonstrates predictions biased by race even though race was not explicitly included in the adaptive boosting. That's because zipcode, a variable highly associated with race, was included in the model. Read the complete vignette to see how Paige modified the model to ameliorate that bias, while still maintaining its predictive power. All of the associated R code is available in the iPython Notebook.
- http://blog.revolutionanalytics.com/2018/06/understanding-bias.html
- TEMPDB – Files and Trace Flags and Updates, Oh My!
- Object Allocation Contention: There are some cases where having multiple files alone does not completely address PFS contention. For these cases, we have implemented a fix where we not only round robin between the files, we also round robin between the PFS pages within the file. The tradeoff with spreading the allocations throughout the file is that a shrink operation may take longer because it will need to relocate data to the beginning of the file.
- With this change, not only will increasing the number of files help with PFS contention, increasing the size of the files (which increases the number of PFS pages in the file) will also help.
- Metadata Contention: Whenever we remove something from the temp table metadata cache, we need to delete the corresponding rows from our metadata tables. Now we are starting to see contention on the deletes from the cache because of the increase in table metadata that comes along with some of the new features implemented in this version. To address this problem, we have made 3 main changes to how we prune the temp table cache: (1) We changed the process from synchronous to asynchronous. (2) We reduced the number of helper threads to one per NUMA node and increased the number of tables that get removed with each pass. (3) We optimized the latching strategy used when we scan for metadata.
- So how do you know if you have this metadata contention? Rather than PFS and SGAM pages, these will be pages that belong to system objects such as sysobjvalues and sysseobjvalues
- We are continuing to work on improving tempdb performance and metadata contention, but in the meantime, there are some best practices you can employ in your code that might help avoid the contention: (1) Do not explicitly drop temp tables at the end of a stored procedure, they will get cleaned up when the session that created them ends. (2) Do not alter temp tables after they have been created. (3) Do not truncate temp tables (4) Move index creation statements to the new inline index creation syntax that was introduced in SQL Server 2014.
- Auditing Overhead: We introduced functionality to allow for transaction-level auditing in Common Criteria Compliance (CCC) which can cause additional overhead, particularly in workloads that do heavy inserts and updates in temp tables, even if CCC is not enabled. In SQL Server 2016 you can enable trace flag 3427 to bypass this overhead starting with SP1 CU2. Starting in SQL Server 2017 CU4, we automatically bypass this code if CCC is disabled.
- https://blogs.msdn.microsoft.com/sql_server_team/tempdb-files-and-trace-flags-and-updates-oh-my/
- Lift SQL Server Integration Services packages to Azure with Azure Data Factory
- Azure Data Factory capabilities are generally available for customers to easily lift SSIS packages to Azure gaining scalability, high availability, and lower TCO, while ADF manages resources for them.
- Azure-SSIS Integration Runtime (IR) are dedicated ADF servers for SSIS package executions.
- Provides the ability to host catalog of SSIS projects (SSISDB) in Azure SQL Managed Instance or Azure SQL Database with a VNet service endpoint.
- Data engineers and developers can continue to use familiar SQL Server Data Tools (SSDT) and SQL Server Management Studio (SSMS) to design, deploy, configure, execute, and monitor SSIS packages in the cloud.
- https://azure.microsoft.com/en-us/blog/lift-sql-server-integration-services-packages-to-azure-with-azure-data-factory/
- https://azure.microsoft.com/en-us/blog/azure-data-factory-new-capabilities-are-now-generally-available/
- New memory options for Analysis Services
- Cross-instance point-in-time restore in Azure SQL Database Managed Instance
- Azure SQL Database Managed Instance enables you to create a database as a copy of another database at some point in time in the past. This is known as point-in-time restore feature, and up till now you could perform point-in-time restore only within the same instance.
- The latest release of Azure SQL Database Managed Instance enables you to perform point-in-time restore of a database from one instance to another. This might be useful if you need to be sure that you could easily restore a database to another instance if there is some issue on the original instance, or if you need a database for testing or auditing purposes on the test instance and you want to use copy of some of the existing database on another server.
- https://blogs.msdn.microsoft.com/sqlserverstorageengine/2018/06/07/cross-instance-point-in-time-restore-in-azure-sql-database-managed-instance/
Recent Training and Technical Guides
- Free E-Book: A Developer’s Guide to Building AI Applications
- O’Reilly and Microsoft are excited bring you a new e-book on AI
- https://blogs.technet.microsoft.com/machinelearning/2018/06/04/free-e-book-a-developers-guide-to-building-ai-applications/
- AI Lab: Learn to Code with the Cutting-Edge Microsoft AI Platform
- AI Lab is a collection of AI projects designed to help developers explore, experience, learn about and code with the latest Microsoft AI Platform technologies.
- https://blogs.technet.microsoft.com/machinelearning/2018/06/19/ai-lab-learn-about-experience-code-with-the-cutting-edge-microsoft-ai-platform/
- Webinar: Securing your data with Azure SQL Database
- In this webinar, you’ll learn how to use the results from our new vulnerability assessment and data classification tools to devise a security strategy.
- https://cloudblogs.microsoft.com/sqlserver/2018/06/25/securing-your-data-with-azure-sql-database/
- Webinar: My Power BI report is slow: what should I do? by Marco Russo
Script and Tool Tips
- MAP Toolkit 9.9 Now Available!
- Create database on Azure SQL Managed Instance using PowerShell
- Quick-start script: Create Azure SQL Managed Instance using PowerShell
- Quick-start script: Setup Azure network environment for Azure SQL Managed Instance
- Point-in-time restore of a database on Azure SQL Managed Instance using AzureRm.Sql PowerShell library
- PowerTip: Ensure that errors in PowerShell are caught
- For Try Catch Finally, you need to make sure the error defaults to a “Stop” action for the cmdlet in question.
- Example: try{ invoke-sqlcmd ....... -ErrorAction stop} catch{ 'SQL Error was caught'}
- https://blogs.technet.microsoft.com/heyscriptingguy/2018/05/31/powertip-ensure-that-errors-in-powershell-are-caught/
- PowerTip: Determine your version of PowerShell and host operating system
- $PSVersionTable.PSVersion -or- $PSVersionTable.OS (on open source release)
- https://blogs.technet.microsoft.com/heyscriptingguy/2018/06/29/powertip-determine-your-version-of-powershell-and-host-operating-system/
- PowerTip: Use PowerShell to play WAV files
- Use the System.Media.Soundplayer object
- https://blogs.technet.microsoft.com/heyscriptingguy/2018/06/15/powertip-use-powershell-to-play-wav-files/
Fany Carolina Vargas | SQL Dedicated Premier Field Engineer | Microsoft Services