Quantcast
Channel: Forum SQL Server Database Engine
Viewing all 15694 articles
Browse latest View live

Permissions problem: Cannot process the object ""databasename"."dbo"."tablename"". The OLE DB provider "SQLNCLI11" for linked server "servername" indicates that either the object has no columns or the current user does not have permissions on that object

$
0
0

I have what I think is a permissions problem in a new database environment.

I have a table T1 in database DB1 on database server DBS1 that user U1 in DBS1 has the ability to read.  It has the db_datareader role for DB1.

I have a linked server LS1 on database server DBS2 that points to DBS1 using the SQL Server account for user U1.

When I run a select query using 4-part naming on DBS2 that … "select * from [LS1].DB1.dbo.T1"

I get an error: 

Cannot process the object ""DB1"."dbo"."T1"". The OLE DB provider "SQLNCLI11" for linked server "LS1" indicates that either the object has no columns or the current user does not have permissions on that object.

At 1st I thought the problem was that I was in SSMS with Windows account which does not have access to DBS1, but I then got the same error connecting to DBS2 via SSMS with the U1 SQL account.  I thought all that mattered was the SQL account in the linked server definition.

We also have scheduled jobs on DBS2 that uses the linked server LS1 every night successfully.



SQL Server 2008 Link Compatibility with Oacle19C

$
0
0

Hi,

I have many DB link from SQL Server 2008 to Oracle11G and vice verse. My company planning to upgrade Oracle from 11G to 19C. Can you please let us know is there any compatibility issue?

or do you have any SQL Server-Oracle compatibility matrix.

Regards,

Deepak

SQL Server Agent Job Notifications Not Working

$
0
0
We have successfully setup Database Mail, and I am able to right click and 'Send Test E-Mail' and it works perfect. We have a few jobs setup in SQL Server Agent, and if the job fails it should e-mail us. Recently a job failed, but no e-mail. So I looked at the job log and see:

NOTE: Failed to notify 'Development' via email.

I confirm under Operators we have `Development` with the e-mail address I would like the error message to go to. Any ideas what would cause this?

System_health XE reports

How to change the MSSQLSERVER Service Startup Account from NT Service\MSSQLSERVER to custom Windows account ?

$
0
0

Hello


Can you let me know an easy method to change the MSSQLSERVER Service Startup Account from NT Service\MSSQLSERVER to a custom Windows account in 2016 version ? Here are my requirements.


1) Create a new Windows O.S account called sqlserver_owner.

2) But do Not make it part of the administrators group.

3) Create an SQL Server login principle with SYSADMIN fixed server role for this sqlserver_owner account. 

I went to SQL Server configuration manager and chosen the SQL Server (MSSQLSERVER) account => right click and tried to change from NT Service\MSSQLSERVER to sqlserver_owner. It changed it. 

4) Now when i login to the Windows server using sqlserver_owner, i am not able to stop and start this service. When i try from windows component services, the stop and start buttons are greyed out.  When i try to click on SSMS => right click => restart , it asks for Administrator's password.

Basically, i would like to use a custom Windows login name as the MSSQLSERVER service owner, but not make it part of Windows administrators group.  Is it not possible in SQL server 2016 version ?

For audit compliance databases, i do not want to make it part of Windows administrators group for segregation of duties. The DBA login should not be part of Windows administrators group

I have seen this article https://msdn.microsoft.com/en-us/library/ms143504.aspx , but it has several steps and a series of permissions to be granted to sqlserver_owner to make it equal in nature to NT Service\MSSQLSERVER . But that is very lengthy and I hoping for a simple solution.

In older versions such as 2008, all that i had to do was add to SQLServerMSSQLUser$ComputerName$InstanceName and SQLServerSQLAgentUser$ComputerName$InstanceName groups.

-Srini


How much RAM should I dedicate to a SQL instance on a server(Question about SQLmax Formula)?

$
0
0

Hi experts,

  I have a dedicated Server(HPE DL580) with 3TB DRAM. According to https://sqlmax.chuvash.eu/, I should allocated 2,649,800 MB to the dedicated SQL Server. But is it right choice? I have 423GB available memory constantly and I think maybe I can allocate more memory to SQL Server, e.g. 2.8TB and leave maybe 200GB for OS is enough.

  Where is this formula coming from? I can't find any Microsoft documents mentioning this?

----

SQL Max Memory = TotalPhyMem - (NumOfSQLThreads * ThreadStackSize) - (1GB * CEILING(NumOfCores/4))
NumOfSQLThreads = 256 + (NumOfProcessors*- 4) * 8 (* If NumOfProcessors > 4, else 0)
ThreadStackSize = 2MB on x64 or 4 MB on 64-bit (IA64)



Trying to understand Trasaction behavior

$
0
0
Hi All,

Today, I got a case related to transaction behavior on SQL Server.
One of our Informatica ETL developer came back saying there is some problem with the sql database.
I went on call to try to understand what she was saying.

Problem description:
They have one table and try to load some data using Informatica ETL tool.
Using Informatica ETL Tool, they have written a simple transformation logic to load data. (truncate & load).

Table structure
==============
create table test_tbl
(
    id         varchar(100) null,
    last_update_date     datetime2(7) null
)
create unique clustered index pk_test_dbl on test_tbl(id asc);
go

For sake of testing, they are loading 1000 recs and trying to load 2 additional records (intentionally to reproduce a duplicate record).

Something like below

Truncate table test_tbl;

Insert into test_tbl
select top 1000
c1,c2 from srcdb.dbo.stg_tbl
union
select '9999','2019-10-30 00:00:00'
union
select '9999','2019-10-30 00:00:00'     --- simulate a duplicate record


Questions :
scenario 1 : when they run the ETL Tool pointing to dev server ,dev db, they see expected behaviour i.e. 1001 rows loaded and 1 rec is rejected as it is a duplicate 1. (i.e. 9999 )
scenario 2: when they change the connection, this time pointing prod server, prod db, they seeing different behaviour. They see only 700+ rows are getting inserted in prod table and not 1001.
                    they were asking why? it's the same truncate and load , works in dev perfectly and why it is not working the same for prod db?

To isolate the issue, I want to do some tests in SSMS, to remove the informatica tool out of picture.

CASE 1:
Ran below statements against dev and prod, I see the transaction behaviour as same (. i.e. SQL is considering the whole thing as single transaction, even 1 record fails, the entire txn is rollbacked. )

truncate table test_tbl;
go

Insert into test_tbl
select top 1000
c1,c2 from srcdb.dbo.stg_tbl
union
select '9999','2019-10-30 00:00:00'
union
select '9999','2019-10-30 00:00:00'     --- simulate a duplicate record

Msg 2601, Level 14, State 1, Line 11
Cannot insert duplicate key row in object 'dbo.test_tbl' with unique index 'pk_test_dbl'. The duplicate key value is (9999).

select * from test_tbl
--no rows


CASE 2 : I have trunacted the table and try to insert only 2 rows . here, this time 1 is getting inserted and 1 row rejected. Infact, i was expecting above error as CASE 1.
truncate table test_tbl
go


Insert into test_tbl
select '9999','2019-10-30 00:00:00'
union
select '9999','2019-10-30 00:00:00'     --- simulate a duplicate record

(1 row(s) affected)

select * from test_tbl
id        last_update_date
9999    2019-10-30 00:00:00.0000000


Can anyone explain this transaction behavior of sql server and ssms ? What is the batch size in sql server to commit or rollback? The reason why I am asking this is, If I try to insert 2 rows , I don't see any error. why ????
Once I get know the exact sql behaviour then I go back to the team and tell this is how sql server treats batch's and if there is something needs to be tweaked in terms of batch size or any db setting in inform them accordingly.

Environment details
==================
select @@version
Microsoft SQL Server 2012 (SP4) (KB4018073) - 11.0.7001.0 (X64)
    Aug 15 2017 10:23:29
    Copyright (c) Microsoft Corporation
    Enterprise Edition: Core-based Licensing (64-bit) on Windows NT 6.3 <X64> (Build 9600: ) (Hypervisor)



Thanks,
Sam

Error - Wait on the Database Engine recovery handle failed. Check the SQL Server error log for potential causes.

$
0
0

Hello team,

I have failed to install SQL server 2017 both developer edition and express,

I have tried very hard to track various sources online to solve this problem for three days but I have failed find a solution.

I have struggled to uninstall all previous failed mssql installations, to delete mssql server folders manually and to reinstall but I am stuck at the same point.

Anyone with advice or solution to this problem please share it here.

Below is my error log

Overall summary:
  Final result:                  Failed: see details below
  Exit code (Decimal):           -2061893606
  Start time:                    2019-10-31 07:03:05
  End time:                      2019-10-31 07:26:59
  Requested action:              Install

Setup completed with required actions for features.
Troubleshooting information for those features:
  Next step for FullText:        Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Next step for sql_inst_mpy:    Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Next step for sql_inst_mr:     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Next step for AdvancedAnalytics: Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Next step for SQLEngine:       Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Next step for Replication:     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.


Machine Properties:
  Machine name:                  HIDDEN
  Machine processor count:       8
  OS version:                    Microsoft Windows Server 2012 R2 Datacenter (6.3.9600)
  OS service pack:               
  OS region:                     United States
  OS language:                   English (United States)
  OS architecture:               x64
  Process architecture:          64 Bit
  OS clustered:                  No

Product features discovered:
  Product              Instance             Instance ID                    Feature                                  Language             Edition              Version         Clustered  Configured

Package properties:
  Description:                   Microsoft SQL Server 2017 
  ProductName:                   SQL Server 2017
  Type:                          RTM
  Version:                       14
  SPLevel:                       0
  Installation location:         C:\SQLServer2017Media\Developer_ENU\x64\setup\
  Installation edition:          Express

Product Update Status:
  None discovered.

User Input Settings:
  ACTION:                        Install
  ADDCURRENTUSERASSQLADMIN:      false
  AGTSVCACCOUNT:                 NT AUTHORITY\NETWORK SERVICE
  AGTSVCPASSWORD:                *****
  AGTSVCSTARTUPTYPE:             Disabled
  ASBACKUPDIR:                   Backup
  ASCOLLATION:                   Latin1_General_CI_AS
  ASCONFIGDIR:                   Config
  ASDATADIR:                     Data
  ASLOGDIR:                      Log
  ASPROVIDERMSOLAP:              1
  ASSERVERMODE:                  TABULAR
  ASSVCACCOUNT:                  <empty>
  ASSVCPASSWORD:                 <empty>
  ASSVCSTARTUPTYPE:              Automatic
  ASSYSADMINACCOUNTS:            <empty>
  ASTELSVCACCT:                  <empty>
  ASTELSVCPASSWORD:              <empty>
  ASTELSVCSTARTUPTYPE:           0
  ASTEMPDIR:                     Temp
  BROWSERSVCSTARTUPTYPE:         Disabled
  CLTCTLRNAME:                   <empty>
  CLTRESULTDIR:                  <empty>
  CLTSTARTUPTYPE:                0
  CLTSVCACCOUNT:                 <empty>
  CLTSVCPASSWORD:                <empty>
  CLTWORKINGDIR:                 <empty>
  COMMFABRICENCRYPTION:          0
  COMMFABRICNETWORKLEVEL:        0
  COMMFABRICPORT:                0
  CONFIGURATIONFILE:             C:\Program Files\Microsoft SQL Server\140\Setup Bootstrap\Log\20191031_070259\ConfigurationFile.ini
  CTLRSTARTUPTYPE:               0
  CTLRSVCACCOUNT:                <empty>
  CTLRSVCPASSWORD:               <empty>
  CTLRUSERS:                     <empty>
  ENABLERANU:                    true
  ENU:                           true
  EXTSVCACCOUNT:                 NT Service\MSSQLLaunchpad
  EXTSVCPASSWORD:                <empty>
  FEATURES:                      SQLENGINE, REPLICATION, ADVANCEDANALYTICS, SQL_INST_MR, SQL_INST_MPY, FULLTEXT, CONN, BC, SDK, SNAC_SDK, LOCALDB
  FILESTREAMLEVEL:               0
  FILESTREAMSHARENAME:           <empty>
  FTSVCACCOUNT:                  NT Service\MSSQLFDLauncher
  FTSVCPASSWORD:                 <empty>
  HELP:                          false
  IACCEPTPYTHONLICENSETERMS:     true
  IACCEPTROPENLICENSETERMS:      true
  IACCEPTSQLSERVERLICENSETERMS:  true
  INDICATEPROGRESS:              false
  INSTALLSHAREDDIR:              C:\Program Files\Microsoft SQL Server\
  INSTALLSHAREDWOWDIR:           C:\Program Files (x86)\Microsoft SQL Server\
  INSTALLSQLDATADIR:             <empty>
  INSTANCEDIR:                   C:\Program Files\Microsoft SQL Server\
  INSTANCEID:                    MSSQLSERVER
  INSTANCENAME:                  MSSQLSERVER
  ISMASTERSVCACCOUNT:            NT AUTHORITY\Network Service
  ISMASTERSVCPASSWORD:           <empty>
  ISMASTERSVCPORT:               8391
  ISMASTERSVCSSLCERTCN:          <empty>
  ISMASTERSVCSTARTUPTYPE:        Automatic
  ISMASTERSVCTHUMBPRINT:         <empty>
  ISSVCACCOUNT:                  NT AUTHORITY\Network Service
  ISSVCPASSWORD:                 <empty>
  ISSVCSTARTUPTYPE:              Automatic
  ISTELSVCACCT:                  <empty>
  ISTELSVCPASSWORD:              <empty>
  ISTELSVCSTARTUPTYPE:           0
  ISWORKERSVCACCOUNT:            NT AUTHORITY\Network Service
  ISWORKERSVCCERT:               <empty>
  ISWORKERSVCMASTER:             <empty>
  ISWORKERSVCPASSWORD:           <empty>
  ISWORKERSVCSTARTUPTYPE:        Automatic
  MATRIXCMBRICKCOMMPORT:         0
  MATRIXCMSERVERNAME:            <empty>
  MATRIXNAME:                    <empty>
  MRCACHEDIRECTORY:              
  NPENABLED:                     0
  PBDMSSVCACCOUNT:               NT AUTHORITY\NETWORK SERVICE
  PBDMSSVCPASSWORD:              <empty>
  PBDMSSVCSTARTUPTYPE:           Automatic
  PBENGSVCACCOUNT:               NT AUTHORITY\NETWORK SERVICE
  PBENGSVCPASSWORD:              <empty>
  PBENGSVCSTARTUPTYPE:           Automatic
  PBPORTRANGE:                   16450-16460
  PBSCALEOUT:                    false
  PID:                           *****
  QUIET:                         false
  QUIETSIMPLE:                   false
  ROLE:                          
  RSINSTALLMODE:                 DefaultNativeMode
  RSSVCACCOUNT:                  <empty>
  RSSVCPASSWORD:                 <empty>
  RSSVCSTARTUPTYPE:              Automatic
  SAPWD:                         *****
  SECURITYMODE:                  SQL
  SQLBACKUPDIR:                  <empty>
  SQLCOLLATION:                  SQL_Latin1_General_CP1_CI_AS
  SQLSVCACCOUNT:                 NT Service\MSSQLSERVER
  SQLSVCINSTANTFILEINIT:         false
  SQLSVCPASSWORD:                *****
  SQLSVCSTARTUPTYPE:             Automatic
  SQLSYSADMINACCOUNTS:           HIDDEN
  SQLTELSVCACCT:                 NT Service\SQLTELEMETRY
  SQLTELSVCPASSWORD:             <empty>
  SQLTELSVCSTARTUPTYPE:          Automatic
  SQLTEMPDBDIR:                  <empty>
  SQLTEMPDBFILECOUNT:            1
  SQLTEMPDBFILEGROWTH:           64
  SQLTEMPDBFILESIZE:             8
  SQLTEMPDBLOGDIR:               <empty>
  SQLTEMPDBLOGFILEGROWTH:        64
  SQLTEMPDBLOGFILESIZE:          8
  SQLUSERDBDIR:                  <empty>
  SQLUSERDBLOGDIR:               <empty>
  SUPPRESSPRIVACYSTATEMENTNOTICE: false
  TCPENABLED:                    0
  UIMODE:                        Normal
  UpdateEnabled:                 true
  UpdateSource:                  MU
  USEMICROSOFTUPDATE:            true
  X86:                           false

  Configuration file:            C:\Program Files\Microsoft SQL Server\140\Setup Bootstrap\Log\20191031_070259\ConfigurationFile.ini

Detailed results:
  Feature:                       Full-Text and Semantic Extractions for Search
  Status:                        Failed
  Reason for failure:            An error occurred for a dependency of the feature causing the setup process for the feature to fail.
  Next Step:                     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Component name:                SQL Server Database Engine Services Instance Features
  Component error code:          0x851A001A
  Error description:             Wait on the Database Engine recovery handle failed. Check the SQL Server error log for potential causes.
  Error help link:               https://go.microsoft.com/fwlink?LinkId=20476&ProdName=Microsoft+SQL+Server&EvtSrc=setup.rll&EvtID=50000&ProdVer=14.0.1000.169&EvtType=0xD15B4EB2%400x4BDAF9BA%401306%4026&EvtType=0xD15B4EB2%400x4BDAF9BA%401306%4026

  Feature:                       Python
  Status:                        Failed
  Reason for failure:            An error occurred for a dependency of the feature causing the setup process for the feature to fail.
  Next Step:                     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Component name:                Machine Learning Services (In-Database)
  Component error code:          0x80131509
  Error description:             Cannot find group with identity S-1-5-21-4006598571-222747567-1642966512-1136.
  Error help link:               https://go.microsoft.com/fwlink?LinkId=20476&ProdName=Microsoft+SQL+Server&EvtSrc=setup.rll&EvtID=50000&ProdVer=14.0.1000.169&EvtType=0x69EA8169%400xE9BC3D64&EvtType=0x69EA8169%400xE9BC3D64

  Feature:                       R
  Status:                        Failed
  Reason for failure:            An error occurred for a dependency of the feature causing the setup process for the feature to fail.
  Next Step:                     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Component name:                Machine Learning Services (In-Database)
  Component error code:          0x80131509
  Error description:             Cannot find group with identity S-1-5-21-4006598571-222747567-1642966512-1136.
  Error help link:               https://go.microsoft.com/fwlink?LinkId=20476&ProdName=Microsoft+SQL+Server&EvtSrc=setup.rll&EvtID=50000&ProdVer=14.0.1000.169&EvtType=0x69EA8169%400xE9BC3D64&EvtType=0x69EA8169%400xE9BC3D64

  Feature:                       Machine Learning Services (In-Database)
  Status:                        Failed
  Reason for failure:            An error occurred during the setup process of the feature.
  Next Step:                     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Component name:                Machine Learning Services (In-Database)
  Component error code:          0x80131509
  Error description:             Cannot find group with identity S-1-5-21-4006598571-222747567-1642966512-1136.
  Error help link:               https://go.microsoft.com/fwlink?LinkId=20476&ProdName=Microsoft+SQL+Server&EvtSrc=setup.rll&EvtID=50000&ProdVer=14.0.1000.169&EvtType=0x69EA8169%400xE9BC3D64&EvtType=0x69EA8169%400xE9BC3D64

  Feature:                       Database Engine Services
  Status:                        Failed
  Reason for failure:            An error occurred during the setup process of the feature.
  Next Step:                     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Component name:                SQL Server Database Engine Services Instance Features
  Component error code:          0x851A001A
  Error description:             Wait on the Database Engine recovery handle failed. Check the SQL Server error log for potential causes.
  Error help link:               https://go.microsoft.com/fwlink?LinkId=20476&ProdName=Microsoft+SQL+Server&EvtSrc=setup.rll&EvtID=50000&ProdVer=14.0.1000.169&EvtType=0xD15B4EB2%400x4BDAF9BA%401306%4026&EvtType=0xD15B4EB2%400x4BDAF9BA%401306%4026

  Feature:                       SQL Server Replication
  Status:                        Failed
  Reason for failure:            An error occurred for a dependency of the feature causing the setup process for the feature to fail.
  Next Step:                     Use the following information to resolve the error, uninstall this feature, and then run the setup process again.
  Component name:                SQL Server Database Engine Services Instance Features
  Component error code:          0x851A001A
  Error description:             Wait on the Database Engine recovery handle failed. Check the SQL Server error log for potential causes.
  Error help link:               https://go.microsoft.com/fwlink?LinkId=20476&ProdName=Microsoft+SQL+Server&EvtSrc=setup.rll&EvtID=50000&ProdVer=14.0.1000.169&EvtType=0xD15B4EB2%400x4BDAF9BA%401306%4026&EvtType=0xD15B4EB2%400x4BDAF9BA%401306%4026

  Feature:                       SQL Browser
  Status:                        Passed

  Feature:                       SQL Writer
  Status:                        Passed

  Feature:                       LocalDB
  Status:                        Passed

  Feature:                       Client Tools SDK
  Status:                        Passed

  Feature:                       Client Tools Connectivity
  Status:                        Passed

  Feature:                       Client Tools Backwards Compatibility
  Status:                        Passed

  Feature:                       SQL Client Connectivity
  Status:                        Passed

  Feature:                       SQL Client Connectivity SDK
  Status:                        Passed

  Feature:                       Setup Support Files
  Status:                        Passed

Rules with failures:

Global rules:

Scenario specific rules:

Rules report file:               C:\Program Files\Microsoft SQL Server\140\Setup Bootstrap\Log\20191031_070259\SystemConfigurationCheck_Report.htm


Cannot open database version 782. upgrade the database to the latest version.

$
0
0

Hi,

I'm trying to restore a SQL Server 2014 database backup on SQL server 2016. The DB restore commands works fine but when I try to issue SQL queries on SQL Server 2016 DB, I get following errors:
Cannot open database version 782. upgrade the database to the latest version.

It seems the DB is unable to upgrade the internal database version from 782 to 852 when we try to bring the database online using "alter database [DB_name] set online" command.

[HELP!!] sqlservr.exe high memory usage - after changing AWE MEMORY LIMIT!!

$
0
0

Hey!
Just noticed my sqlservr.exe service has huge memory usage (almost 8GB RAM) toonly 1 DB
I would like to know how to fix that.. tried to set limit for memory by

(local) -> properties -> memory 


but it doesnt look like there's any difference..

this thing makes my windows crash every day!
I need help ASAP
THANKS!

TempDB question - possible issue

$
0
0

Hi all.

I've just had a nose around my production tempdb database and noticed something odd (in my view anyway)

Using queries on sys.dm_db_session_space_usage and sys.dm_db_task_space_usage, I have found that there are two sessions that were created on the 10th of April that are still being used (as per Last Batch column) but do not release themselves (ie, session gets killed) after the batch is completed. It seems it is also growing with each execution, as per the highlighted allocation columns in the attachment.

With this comes two questions:

  1. Should I be concerned about these sessions not being killed after each execution...?
  2. What is the difference between allocated and deallocated mb? These columns reference the internal_objects_alloc_page_count and internal_objects_dealloc_page_count in sys.dm_db_session_space_usage DMV

Any questions, please let me know.

Thanks in advance.

Sort operator in Execution Plan

$
0
0

I have following simple query.

SELECT STT.SWallet_ID
FROM dbo.TRANSACTION STT 
WHERE STT.Keyword='apmt'
ORDER BY STT.SWallet_ID

Because of order by clause, the execution plan shows sort operator. To remove the sort, I tried various indexes bout could not. According to BOL, sort should be removed by using index on column stt.SWallet_ID.

Backup suddenly very slow

$
0
0

Hi all,

Working with a Microsoft SQL Server 2017 (RTM-CU17) instance, we had a system (non SQL Server related) failure. After that, we recovered the machines but still have a big issue with one SQL instance (we have three instances on the same virtual machine).

The backups of one instance run very slow. In the past days we need just three hours for both the databases of the instance, now looks like we need about three days. Both databases are not so big (3/5 GB) but have lots of Filestream documents.

I tried to change backup parameters, to backup on a Nul disk, to create a new, small db and backup it (with no issue) , to check the configuration and so on.

Just I noticed an high CPU consumption and the fact that the backup hangs more than 10 minutes with ASYNC_IO_COMPLETION wait, and then starts (very very slow) with BACKUPTHREAD wait. Performance dashboard shows the cache hit ratio collapsing.

Any idea?

Thanks in advance.



Luca SQL DBA

Polybase External Table Creation Fails with ODBC Driver Data Source in SQL SERVER 2019 but works with Linked Server

$
0
0

          

We are trying to create a External Table to Connect to the Databricks Delta Table via Simba Spark ODBC Driver Data Source. It is failing with the below error. The schema is matching with the source, but still we are getting this error. We have Validated the same with the Linked Server via the same ODBC Driver Data Source, we were able to Fetch the Data. Also we have traced the External Table Creation Call in the Databricks and we did not see any failure and it has retured the Schema Details to the Caller.

Error:

Msg 105083, Level 16, State 1, Line 1

105083;The following columns in the user defined schema are incompatible with the external table schema for table 'PriceList4': user defined column: 'PriceListKey' was not found in the external table. The detected external table schema is: ([PriceListKey] INT).


Could you please help us to resolve the Issue.

CREATE EXTERNAL DATA SOURCE DatabricksSource
WITH ( LOCATION = 'odbc://localhost',
CONNECTION_OPTIONS = 'DSN=lire2ewaep',
 PUSHDOWN = ON,
CREDENTIAL = ODBCCredential );


CREATE EXTERNAL TABLE PriceList4
([PriceListKey] INT )
WITH (DATA_SOURCE = DatabricksSource,
LOCATION = '[pricelist4]'
);





Polybase SQL Server 2019 : External Table Query Result Issue

$
0
0

We have couple of CSV files on Azure Container Folder and accessing those files with Polybase External Table.

All the files has the same structure, data of each file can be queried when accessed  separately. 

CREATE EXTERNAL TABLE [dbo].[dbs_ACC_DATA]
(

[Account Number] [varchar](50) NULL,[Client ID] [varchar](100) NULL,[Active?] [varchar] (10) NULL,[Individual or Entity Account] [varchar] (10) NULL)

WITH (DATA_SOURCE = [AzureDS],LOCATION = N'/client_files_dbs/file_1.csv',FILE_FORMAT =[az_storage_files],REJECT_TYPE = VALUE,REJECT_VALUE = 1)

SELECT * FROM dbs_ACC_DATA

We are running into the following issues when accessing all the files in the folder

CREATE EXTERNAL TABLE [dbo].[dbs_ACC_DATA]
(

[Account Number] [varchar](50) NULL,[Client ID] [varchar](100) NULL,[Active?] [varchar] (10) NULL,[Individual or Entity Account] [varchar] (10) NULL)

WITH (DATA_SOURCE = [AzureDS],LOCATION = N'/client_files_dbs/',FILE_FORMAT =[az_storage_files],REJECT_TYPE = VALUE,REJECT_VALUE = 1)

Error : 

Cannot execute the query "Remote Query" against OLE DB provider "MSOLEDBSQL" for linked server "(null)". 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed.

Error log from DWEngine_server log file. 

CREATE EXTERNAL TABLE [dbo].[dbs_ACC_DATA]
(
	[Account Number] [varchar](50) NULL,	[Client ID] [varchar](100) NULL,	[Active?] [varchar] (10) NULL,	[Individual or Entity Account] [varchar] (10) NULL,	[Single or Joint Account] [varchar] (10) NULL,
	[Number of Account Holders] [varchar] (10) NULL,	[Entity Name] [varchar](300) NULL,	[DBA/Disregarded Entity Name] [varchar](100) NULL,	[Country of Organization/Incorporation] [varchar](2) NULL,
	[First Name] [varchar](300) NULL,	[Middle Name] [varchar](300) NULL,	[Last Name] [varchar](300) NULL,	[Citizenship Country] [varchar](2) NULL,	[Country of Birth] [varchar](2) NULL,	[Date of Birth] [varchar](100) NULL,
	[PR Address - Street 1] [varchar](150) NULL,	[PR Address - Street 2] [varchar](150) NULL,	[PR Address - City] [varchar](100) NULL,	[PR Address - Postal Code] [varchar](50) NULL,	[PR Address - State] [varchar](100) NULL,	[PR Address - Country] [varchar](2) NULL,
	[Mailing Address - Street 1] [varchar](150) NULL,	[Mailing Address - Street 2] [varchar](150) NULL,	[Mailing Address -  City] [varchar](100) NULL,	[Mailing Address - Postal Code] [varchar](50) NULL,	[Mailing Address - State] [varchar](100) NULL,
	[Mailing Address - Country] [varchar](2) NULL,	[Additional Mailing Address Countries] [nvarchar](100) NULL,	[Phone Number] [nvarchar](30) NULL,	[Standing Instructions Country] [varchar](2) NULL,
	[Account Opening Date] [datetime] NULL,	[Classification of a U.S. Person] [int] NULL,	[Foreign TIN Type] [nvarchar](6) NULL,
	[U.S. TIN] [varchar](9) NULL,	[U.S. TIN Type] [nvarchar](6) NULL,	[Registration Code] [nvarchar](100) NULL,	[Email Address] [varchar](200) NULL,	[Title] [varchar](100) NULL,
	[Town of Birth] [varchar](50) NULL,	[Reporting FI Jurisdiction] [varchar](2) NULL,	[Tax Residence] [nvarchar](2) NULL,	[POA Country] [varchar](2) NULL,
	[Hold Mail Country] [varchar](2) NULL,	[CIC Code] [int] NULL,	[FILER1] [varchar](100) NULL,	[FILER2] [varchar](100) NULL,	[FILER3] [varchar](100) NULL,	[FILER4] [varchar](100) NULL,	[FILER5] [varchar](100) NULL,
	[FILER6] [varchar](100) NULL,	[FILER7] [varchar](100) NULL,	[FILER8] [varchar](100) NULL,	[FILER9] [varchar](100) NULL,	[FILER10] [varchar](100) NULL
)
WITH (DATA_SOURCE = [AzureDS],LOCATION = N'/client_files_dbs/',FILE_FORMAT =[az_storage_files],REJECT_TYPE = VALUE,REJECT_VALUE = 1)
 SID333:QID507 [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ServerInterface:InformationEvent] (Info, Normal): Starting processor SqlFrontEndWorkProcessor. [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Initializing bridge... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): GetFileMetadata... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [HdfsBridgeJniWrapper:InformationEvent] (Info, Normal): Acquired JvmCreationLock... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): Getting file info for each file in a directory containing total 4 files [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): File info obtained for each file. [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:3652] [ServerInterface:InformationEvent] (Info, Normal): Query SID333:QID507 completed. [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:4960] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID333][Session.IsTransactional:False]
8/23/2019 12:28:10 AM [Thread:4960] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209822992 
8/23/2019 12:28:10 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): CancelBatch request started [Session.SessionId:SID333] 
8/23/2019 12:28:10 AM [Thread:4556] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209824864 
8/23/2019 12:28:10 AM [Thread:4556] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID333][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Incoming TDS connection, Client TDS version: 7 (TDS74). 
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209822992 Client: 192.168.10.49:51536, isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe: 192.168.10.49:51536 
8/23/2019 12:28:12 AM [Thread:4556] [DataClassificationConfig:InformationEvent] (Info, Normal): Returning FeatureSwitch DataClassificationCoreEnabled status: False [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:ServerStartSessionEvent] (Info, Low): Started new session, in reset connection: False. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): Processing login, authentication scheme: "Ntlm", client application name: "87E0B5F8-7B5A-776E-2D52-AA5CB888E28D" 
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Data Classification TDS Extension Requested=False, FS Enabled=False, Enabled=False. [Session.SessionId:SID334][Session.IsTransactional:True][Query.QueryId:QID508]
8/23/2019 12:28:12 AM [Thread:1668] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209824864 Client: , isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe:  
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209824864 
8/23/2019 12:28:12 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): CancelBatch request started [Session.SessionId:SID334] 
8/23/2019 12:28:12 AM [Thread:4636] [EngineInstrumentation:EngineCancelQueryBeginEvent] (Info, Low): Cancel requested. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209822992 
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Incoming TDS connection, Client TDS version: 7 (TDS74). 
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209822992 Client: 192.168.10.49:51537, isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe: 192.168.10.49:51537 
8/23/2019 12:28:12 AM [Thread:4556] [DataClassificationConfig:InformationEvent] (Info, Normal): Returning FeatureSwitch DataClassificationCoreEnabled status: False [Session.SessionId:SID335][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:ServerStartSessionEvent] (Info, Low): Started new session, in reset connection: False. [Session.SessionId:SID335][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): Processing login, authentication scheme: "Ntlm", client application name: "87E0B5F8-7B5A-776E-2D52-AA5CB888E28D" 
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Data Classification TDS Extension Requested=False, FS Enabled=False, Enabled=False. [Session.SessionId:SID335][Session.IsTransactional:True][Query.QueryId:QID509]
8/23/2019 12:28:12 AM [Thread:1668] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209824864 Client: , isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe:  
8/23/2019 12:28:12 AM [Thread:4556] [ServerInterface:InformationEvent] (Info, Normal): Incoming Query:  SID335:QID510 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:12 AM [Thread:3652] [ServerInterface:InformationEvent] (Info, Normal): Starting processor ExecuteMemoProcessor. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): Memo compilation time: 142.5795 ms [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): Sql Server Optimization Clock: 0 s, CPU: 0.002 s. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): SQL Server XML generation Clock : 0.011 s, CPU: 0.01 s [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Initializing bridge... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): GetFileMetadata... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [HdfsBridgeJniWrapper:InformationEvent] (Info, Normal): Acquired JvmCreationLock... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): Getting file info for each file in a directory containing total 4 files [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): File info obtained for each file. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [ReaderHashCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00021. Default: 0.00021 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [WriterCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 2.4E-05. Default: 2.4E-05 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [InsBulkCpyCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00024. Default: 0.00024 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [NetworkCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 1E-09. Default: 1E-09 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [ExternalReadCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 4.8E-05. Default: 4.8E-05 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [ExternalHadoopWriteCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00024. Default: 0.00024 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [StreamWriteCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00024. Default: 0.00024 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [LimitNumberOfScaOpConst:InformationEvent] (Info, Normal): The number of literals in the query: 0. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [DwServiceSoftLimitSchemaEnabled] has been initialized. Source provider: Microsoft.SqlServer.DataWarehouse.Configuration.FeatureSwitchSettingsProvider. Value: False. Default: False [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [DwServiceSoftLimitQueryOptEnabled] has been initialized. Source provider: Microsoft.SqlServer.DataWarehouse.Configuration.FeatureSwitchSettingsProvider. Value: False. Default: False [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): Distributed QO time: 1541.9893 ms [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [ExternalFile:InformationEvent] (Info, Normal): For file format DelimitedText, max # of external readers per node has been set to 8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [ExternalFile:InformationEvent] (Info, Normal): For file format DelimitedText, max # of external readers per node has been set to 8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [HadoopFile:InformationEvent] (Info, Normal): Total # of files: 4, Total file splits: 4 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [ExternalFile:InformationEvent] (Info, Normal): For file format DelimitedText, max # of external readers per node has been set to 8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): [BuildExternalMoveReaders] Total 4 split lists for 1 compute nodes each can use up-to 8 external readers. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): [BuildExternalMoveReaders]
Node [SQLEXT2016M01:1433]; # of readers used: 4; Reader 0, # of splits: 1; Reader 1, # of splits: 1; Reader 2, # of splits: 1; Reader 3, # of splits: 1;
 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [DwDmsHighResourcePlanPercentage] has been initialized. Source provider: Microsoft.SqlServer.DataWarehouse.Configuration.XdbSettingsProvider. Value: -1. Default: -1 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DmsConcurrency:InformationEvent] (Info, Normal): DwDmsHighResourcePlanPercentage is loaded: [-1]. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DmsConcurrency:InformationEvent] (Info, Normal): Check if this instance is DwService Gen1 : [False]. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSManager:InformationEvent] (Info, Normal): ExecuteCommand Query:SID335 Plan:a55cf35f-387a-46de-bb2a-4bf6791c925b [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSManager:InformationEvent] (Info, Normal): DMS Manager starting query: SID335, plan: a55cf35f-387a-46de-bb2a-4bf6791c925b. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:23 AM [Thread:4328] [DMSManager:InformationEvent] (Info, Normal): ExecuteCommand Query:SID335 Plan:61ddf166-0d34-4537-b799-09b2e7f65aa8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:23 AM [Thread:4328] [DMSManager:InformationEvent] (Info, Normal): DMS Manager starting query: SID335, plan: 61ddf166-0d34-4537-b799-09b2e7f65aa8. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:23 AM [Thread:4296] [DMSManager:InformationEvent] (Info, Normal): Handle de-registered PBQTable{0F05B8EF-B910-4F5B-976E-40AF27BF85DA} 
8/23/2019 12:28:27 AM [Thread:4296] [CommandWorker:InformationEvent] (Info, Normal): QueryId QID510 PlanId a55cf35f-387a-46de-bb2a-4bf6791c925b StepId 3: Canceling plan due to reject threshold reached. 
8/23/2019 12:28:27 AM [Thread:3812] [CommandWorker:InformationEvent] (Info, Normal): Cancel triggered for data movement 
8/23/2019 12:28:28 AM [Thread:360] [CommandWorker:ErrorEvent] (Error, High): Plan failed with DMS CommandWorker Exception: 
Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3652] [AbstractDataMovementExecutable`1:ErrorEvent] (Error, High): Data movement error detected- Code: 0
Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3652] [AbstractDataMovementExecutable`1:InformationEvent] (Info, Normal): DMS Manager finishing query: SID335, plan: a55cf35f-387a-46de-bb2a-4bf6791c925b, queryId: QID510 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3652] [EngineInstrumentation:QueryStepExecuteErrorEvent] (Error, Critical): 
Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed. ---> Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed.
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.HandleError()
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.OnExecute(ISessionContext Session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractExecutable`1.Execute(ISessionContext session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.<>c__DisplayClass40_0.<ExecuteExecutable>b__0(IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteActionInDistributedTransactions(Int32 transactionsCount, Action`1 action, IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteExecutable(EngineExecutionPlan plan, IExecutable executable, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecutePlan(EngineExecutionPlan plan, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.RunExecutables(EngineExecutionPlan plan, IList`1 executables)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.MultiStreamingExecutable.OnExecute(ISessionContext session) [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:4328] [AbstractDataMovementExecutable`1:InformationEvent] (Info, Normal): DMS Manager finishing query: SID335, plan: 61ddf166-0d34-4537-b799-09b2e7f65aa8, queryId: QID510 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:4328] [MultiStreamingExecutable:InformationEvent] (Info, Normal): Early terminate other operations in stream. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3812] [CommandWorker:WarningEvent] (Warning, Normal): Cannot cancel this Plan:61ddf166-0d34-4537-b799-09b2e7f65aa8, due to the uncancellable State:Completed 
8/23/2019 12:28:28 AM [Thread:3812] [CommandWorker:InformationEvent] (Info, Normal): Cancel triggered for data movement 
8/23/2019 12:28:28 AM [Thread:3652] [MultiStreamingExecutable:InformationEvent] (Info, Normal): One or more exceptions hit. Generating aggregated exception summary
===========================================
Exception summary
Exception Message: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed.
Exception stack trace
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.HandleError()
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.OnExecute(ISessionContext Session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractExecutable`1.Execute(ISessionContext session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.<>c__DisplayClass40_0.<ExecuteExecutable>b__0(IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteActionInDistributedTransactions(Int32 transactionsCount, Action`1 action, IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteExecutable(EngineExecutionPlan plan, IExecutable executable, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecutePlan(EngineExecutionPlan plan, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()

I would like know the reason for the error, which is not very clear from any of the error log files.


Failed to flush the commit table to disk in dbid 9 due to error 3601

$
0
0

One the database backup is failing with the following error.
I ready tried disabling change tracking which dons not helped. infact chage tracking already disabled.
No errors in DBCC CheckDB

This sever is upgraded from 2008 r2 to 2017

Current SQL version : Microsoft SQL Server 2017 (RTM-CU17) (KB4515579) - 14.0.3238.1 (X64)   Sep 13 2019 15:49:57   Copyright (C) 2017 Microsoft Corporation  Standard Edition (64-bit) on Windows Server 2016 Standard 10.0 <X64> (Build 14393: ) (Hypervisor)  


Msg 201, Level 16, State 4, Procedure sys.sp_flush_commit_table, Line 0

Procedure or function 'sp_flush_commit_table' expects parameter '@rowcount', which was not supplied.

Msg 5901, Level 16, State 1, Line 1

One or more recovery units belonging to database 'PRM' failed to generate a checkpoint. This is typically caused by lack of system resources such as disk or memory, or in some cases due to database corruption. Examine previous entries in the error log for more detailed information on this failure.

Msg 3999, Level 17, State 1, Line 1

Failed to flush the commit table to disk in dbid 9 due to error 3601. Check the errorlog for more information.


Do you want to be acknowledged as SQL Server Database Engine Guru? Submit your work to November 2019 competition!

$
0
0

What is TechNet Guru Competition?

Each month the TechNet Wiki council organizes a contest of the best articles posted that month. This is your chance to be announced as MICROSOFT TECHNOLOGY GURU OF THE MONTH!

One winner in each category will be selected each month for glory and adoration by the MSDN/TechNet Ninjas and community as a whole. Winners will be announced in dedicated blog post that will be published in Microsoft Wiki Ninjas blog, a tweet from the Wiki Ninjas Twitter account, links will be published at Microsoft TNWiki group on Facebook, and other acknowledgement from the community will follow.

Some of our biggest community voices and many MVPs have passed through these halls on their way to fame and fortune.

If you have already made a contribution in the forums or gallery or you published a nice blog, then you can simply convert it into a shared wiki article, reference the original post, and register the article for the TechNet Guru Competition. The articles must be written in November 2019 and must be in English. However, the original blog or forum content can be from beforeNovember 2019.

Come and see who is making waves in all your favorite technologies. Maybe it will be you!


Who can join the Competition?

Anyone who has basic knowledge and the desire to share the knowledge is welcome. Articles can appeal to beginners or discusse advanced topics. All you have to do is to add your article to TechNet Wiki from your own specialty category.


How can you win?

  1. Please copy/Write over your Microsoft technical solutions and revelations to TechNetWiki.
  2. Add a link to your new article on THIS WIKI COMPETITION PAGE (so we know you've contributed)
  3. (Optional but recommended) Add a link to your article at the TechNetWiki group on Facebook. The group is very active and people love to help, you can get feedback and even direct improvements in the article before the contest starts.

Do you have any question or want more information?

Feel free to ask any questions below, or Join us at the official MicrosoftTechNet Wiki groups on facebook. Read More about TechNet Guru Awards.

If you win, people will sing your praises online and your name will be raised as Guru of the Month.


PS: Above top banner came from Kamlesh Kumar.





JAYENDRAN ARUMUGAM

Restore transnational backup file failed - sql 2016

$
0
0

monthly full backup, daily differential backup ,  every 5min transnational 



--restore full backup   succeeded
USE [master]
RESTORE DATABASE [test] FROM  DISK = N'X:\Monthly_Backup\OJCLOUD-SQL-WEB_website_local_FULL.bak' WITH  FILE = 1,  
MOVE N'dbfle' TO N'X:\Monthly_Backup\dbfile\test.mdf',  
MOVE N'dbfle_log' TO N'X:\Monthly_Backup\dbfile\test_log.ldf',  NORECOVERY,  NOUNLOAD,  STATS = 5
GO


--restore fifferential backup   succeeded
RESTORE DATABASE  [test] FROM  DISK = N'X:\Monthly_Backup\db\OJCLOUD-SQL-WEB_website_local_DIFF_20191031_141301.bak' WITH  FILE = 1,  
MOVE N'dbfle' TO N'X:\Monthly_Backup\dbfile\test.mdf',  
MOVE N'dbfle_log' TO N'X:\Monthly_Backup\dbfile\test_log.ldf',  NORECOVERY,  NOUNLOAD,  STATS = 5


--restore Transactional log backup   Failed 
--when i try to restore log getting below error


RESTORE log  [test] FROM  DISK = N'X:\Monthly_Backup\db\OJCLOUD-SQL-WEB_website_local_LOG_20191031_021501.trn' WITH  FILE = 1,    NORECOVERY,  NOUNLOAD,  STATS = 5


--Msg 4326, Level 16, State 1, Line 13
--The log in this backup set terminates at LSN 1301372000010520000001, which is too early to apply to the database. A more recent log backup that includes LSN 1302414000030177600001 can be restored.
--Msg 3013, Level 16, State 1, Line 13
--RESTORE LOG is terminating abnormally.

how to find LSN chain FULL to Diff to TRN  ? 

am using below script? 

DECLARE @db_name VARCHAR(100)
SELECT @db_name = 'website_local'
-- Get Backup History
SELECT TOP (1000) s.database_name
,m.physical_device_name
,CAST(CAST(s.backup_size / 1000000 AS INT) AS VARCHAR(14)) + ' ' + 'MB' AS bkSize
,CAST(DATEDIFF(second, s.backup_start_date, s.backup_finish_date) AS VARCHAR(4)) + ' ' + 'Seconds' TimeTaken
,s.backup_start_date
,CAST(s.first_lsn AS VARCHAR(50)) AS first_lsn
,CAST(s.last_lsn AS VARCHAR(50)) AS last_lsn
,CASE s.[type] WHEN 'D'
THEN 'Full'
WHEN 'I'
THEN 'Differential'
WHEN 'L'
THEN 'Transaction Log'
END AS BackupType
,s.server_name
,s.recovery_model
FROM msdb.dbo.backupset s
INNER JOIN msdb.dbo.backupmediafamily m ON s.media_set_id = m.media_set_id
WHERE s.database_name = @db_name
ORDER BY backup_start_date DESC
,backup_finish_date

Compress backup of database that was born as 2005, lives on 2016 and runs in compatibility = 100

$
0
0
Shouldn't I be able to compress a database backup on SQL Server 2016 SP1, even though the database was created on 2005, backup/restore'd to 2016 and running in Comp = 100 mode?

sql server performance related non-technical question

$
0
0

Hi All,

This is a non-technical question and want to know your sincere opinions on this.

In our org, as DBA support Team, we have enough work performing SQL Installations. migrations, upgrades, server builds, Automation's, DB Refreshes, Security etc... and all of a sudden once in a while, some 'X' app team working in the org, comes back on us and say we need help in tuning these bunch of queries which are running slow in the database? As you people are maintaining the databases, tell us why this query is running slow. These queries used to work or run fine within a min or 2 and from last week these queries are running dead slow? We do ask if any changes happened to the code. The immediate answers from them will be NO changes.

And we have to break our heads. So, I want to know, if you were in such a scenario, what would you ask to the application team or how would you tackle such situation so that everyone knows what they have to do and asking the right questions and right help?

Though it was uncomfortable, I try saying, that I am afraid to help in re-writing these huge business logic queries as the code will be changing for every release and that's something Dev team has to revisit and re-write the queries. However, we can try helping in looking at the execution plans and recommend best practices, missing index or covering indexes, identify key performance killers like index fragmentation issues, was the query is getting blocked, deadlocked, key lookups, cursor usage, usage of unwanted hints NOLOCK etc.

Sometimes I feel like saying direct 'NO' and this is not part of our job. Sometimes, you genuinely don't know how to approach or solve that problem. And sometimes, you feel like helping them in all possible ways, but end up wasting time and effort, you don't get any credit and sometimes even end up in an Escalation on your team saying they haven't helped us on timely basis.

So, I want to know, how will you tackle such situations within an organization if you working alone or managing too many things or getting minimal support from your management etc ... ? Please don't say, quit the job and find a new one :)

I would like to know what kind of right questions you might ask to the app team? How do you set that right expectations so that you don't end up taking things too Personal and affecting good relationships between co-workers or within teams ( like Dba team and Systems Team ).

Thanks,
Sam

Viewing all 15694 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>