Business Intelligence Blogs

View blogs by industry experts on topics such as SSAS, SSIS, SSRS, Power BI, Performance Tuning, Azure, Big Data and much more! You can also sign up to post your own business intelligence blog.

«November 2015»

DirectQuery in Power BI Desktop

In the latest Power BI Desktop a new Preview features was released that now allows you to connect using DirectQuery to either SQL Server or Azure SQL Databases.  DirectQuery is a really neat feature that allows you to point to the live version of the data source rather than importing the data into a data model in Power BI Desktop. 

Normally when you want to get an updated dataset in the Power BI Desktop you would have to manually click the refresh button (this can be automated in the Power BI Service), which would initiate a full reimport of your data.  This refresh could take a variable amount of time depending on how much data your have.  For instance, if you’re refreshing a very large table you may be waiting quite a while to see the newly added data. 

With DirectQuery data imports are not required because you’re always looking at a live version of the data.  Let me show you how it works!

Turning on the DirectQuery Preview

Now, because DirectQuery is still in Preview you must first activate the feature by navigating to File->Options and settings->Options->Preview Features then check DirectQuery for SQL Server and Azure SQL Database


Once you click OK you may be prompted to restart the Power BI Desktop to utilize the feature.

Using DirectQuery in Power BI Desktop

Next make a connection either to an On-Premises SQL Server or Azure SQL database.

Go to the Home ribbon and select Get Data then SQL Server.


Provide your Server and Database names then click OK. ***Do not use a SQL statement.  It is not currently supported with DirectQuery***


From the Navigator pane choose the table(s) you would like to use.  I’m just going to pick the DimProduct table for this example and then click Load.  You could select Edit and that would launch the Query Editor where you could manipulate the extract.  This would allow you to add any business rules needed to the data before visualizing it.


Next you will be prompted to select what you want to connect to the data. Again, Import means the data

Read more

The Big Data Blog Series

Over the last few years I’ve been speaking a lot on the subject of Big Data. I started by giving an intermediate session called “Show Me Whatcha’ Workin’ With”. This session was designed for people who had attended a one hour introductory session that showed you how to load data, to look at possible applications … Continue reading The Big Data Blog Series
Read more

Exporting Data Using BCP

  • 20 August 2012
  • Author: BradSchacht
  • Number of views: 1135

BCP, or bulk copy program, has been around in SQL Server for a long time. It is a great way to export large quantities of data very quickly from SQL Server. It can be used to export entire tables or even a custom query. In this post we will focus on doing some simple commands to export data. This is by no means a complete and comprehensive look at BCP. My only intention with this post is to get you started and maybe provide a quick syntax reference for doing BCP in the future.

The obvious choice to move data out to a flat file may be SSIS. BCP could not be any more of a polar opposite in actually implementing the data export in that SSIS is a completely developed in a GUI and BCP is completely developed at the command prompt.

To get started open up a new command prompt window on your computer. You may want to go ahead and run it as administrator in case you want to put the file someplace like the C:\ drive and have UAC turned on. If you simply type BCP at the command prompt a series of available commands will be displayed. We will only touch on a couple of those:

You can see in the screenshot the basic syntax for BCP is [What to Export] [in or out] [file]

BCP AdventureWorks.Production.Product out C:\Production_Product.txt

The second parameter can actually have one of four values:

  • in - Copy data into a table or view from a file
  • out - Copy data from a table or view to a file
  • queryout - Copy data from a query to a file (query must be provided enclosed in quotes, not a table or view name)
  • format - Creates a format file based on the table, view or query specified

There are several other parameters you will want to be sure to include keeping in mind each is case sensitive:

  • -S ServerName\InstanceName
  • Authentication
    • -U Username and -P Password
    • -T Windows Authentication
  • Data Types
    • -c Character data types
    • -n Uses the native data types from the source system
    • Do not specify anything and you will be prompted to provide a data type for each column

Now for a couple of samples:

For each of these I am going to be using windows authentication to connect to a named instance on my local machine. This can connect to other servers however and the files can also be sent to a network share as well.

Table Export: BCP AdventureWorks.Production.Product OUT C:\ProductionProduct.txt -S localhost\SQL2008R2 -T -c

Notice that 504 rows were exported in well under a second at 504,000 rows per second.

Query Export: BCP "SELECT * FROM AdventureWorks.Production.ProductModel" QUERYOUT C:\ProductionProductModel.txt -S localhost\SQL2008R2 -T -c

The results in this case are similar: 128 rows exported, still under a second and 1,376 rows per second.

Notice what happens when providing a query and specifying OUT instead of QUERYOUT:

Unfortunately the error is not all that great, we are just told that an error occurred while processing the command line.

Just to show what happens when exporting a table with more than just a couple of records here is a screenshot of the export from ContosoRetailDW.dbo.FactOnlineSales

Hope this sheds some light on exporting data with BCP. Be on the lookout for some information coming on importing data with BCP as well as a performance comparison between BCP and SSIS direct to a flat file.

Want to explore the BCP options in more detail? Head over to the MSDN page:

Categories: Big Data, SQL Server
Rate this article:
No rating


Other posts by BradSchacht

Please login or register to post comments.