代码之家 › 专栏 › 技术社区 › Akedren

在Azure Automation Runbook中运行时,提交AdlJob失败,找不到方法

azure-automation u-sql azure-data-lake json.net powershell

Akedren · 技术社区 · 8 年前

我正在尝试安排一些我以前通过Azure自动化服务在Azure Data Lake Analytics上运行的U-SQL脚本(我也尝试在ADF中执行此操作,但它对我也不起作用,我将起草另一个问题来解决该路径)。

这是基于 MS Docs Guidance 对我来说仍然存在错误:

$Conn = Get-AutomationConnection -Name AzureRunAsConnection
Connect-AzureRmAccount -ServicePrincipal `
 -Tenant $Conn.TenantID `
 -ApplicationId $Conn.ApplicationID `
 -CertificateThumbprint $Conn.CertificateThumbprint

$adla = "example_adla01"

$script = @"
@a  = 
    SELECT * FROM 
    (VALUES
        ("Contoso", 1500.0),
        ("Woodgrove", 2700.0)
    ) AS D( customer, amount );
OUTPUT @a
TO "/data.csv"
USING Outputters.Csv();
"@

Submit-AdlJob -AccountName $adla -Script $script -Name "Demo"

正在调用的命令的版本。

CommandType  Name            Version   Source
-----------  ----            -------   ------                                    
Alias        Submit-AdlJob   4.2.3     AzureRM.DataLakeAnalytics

这项工作在我的本地Powershell ISE和Cloud Shell中运行良好;但在Azure Automation Service测试运行程序中失败,并出现此错误(已更新以包含更详细的错误):

Submit-AzureRmDataLakeAnalyticsJob : A command that prompts the user failed because the host program or the command type does not support user interaction. The host was attempting to request confirmation with the following message: 

Method not found: 'Newtonsoft.Json.Serialization.IAttributeProvider 
Newtonsoft.Json.Serialization.JsonProperty.get_AttributeProvider()'.
At line:23 char:1

+ Submit-AzureRmDataLakeAnalyticsJob -AccountName $adla -Script $script ...    
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotImplemented: (:) [Submit-AzureRmDataLakeAnalyticsJob], HostException
+ FullyQualifiedErrorId : HostFunctionNotImplemented,Microsoft.Azure.Commands.DataLakeAnalytics.SubmitAzureDataLakeAnalyticsJob

我已经导入了我认为正确的模块,并且找不到任何其他与JSON相关的模块,或者根本找不到调用JSON序列化程序的原因,否则这是我今天早上站起来的一个全新的自动化实例。

命令的调试输出:

7:04:53 PM - SubmitAzureDataLakeAnalyticsJob begin processing with ParameterSet 'SubmitUSqlJob'.

7:04:53 PM - using account id 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'...
[Common.Authentication]: Authenticating using Account: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', environment:  'AzureCloud', tenant: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 

04/23/2018 19:04:54:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenForClientHandler: === Token Acquisition started:
    Authority: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/
    Resource: https://management.core.windows.net/
    ClientId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    CacheType: Microsoft.Azure.Commands.Common.Authentication.AuthenticationStoreTokenCache (1 items)
    Authentication Target: Client


Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Looking up cache for a token...
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - TokenCache: An item matching the requested resource was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: 59.9101559033333 minutes left until token in cache expires
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: A matching item (access token or refresh token or both) was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 :

04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenHandlerBase: === Token Acquisition finished 
successfully. An access token was retuned:
    Access Token Hash: //removed
    Refresh Token Hash: [No Refresh Token]
    Expiration Time: 04/23/2018 20:04:48 +00:00
    User Hash: null


Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54:  - AuthenticationStoreTokenCache: Serializing token cache with 1 items.

[Common.Authentication]: Authenticating using Account: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', environment: 'AzureCloud', tenant: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 
04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenForClientHandler: === Token Acquisition started:
    Authority: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/
    Resource: https://datalake.azure.net/
    ClientId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    CacheType: Microsoft.Azure.Commands.Common.Authentication.AuthenticationStoreTokenCache (1 items)
    Authentication Target: Client
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Looking up cache for a token...
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: No matching token was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Storing token in the cache...
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: An item was stored in the cache

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 :

04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenHandlerBase: === Token Acquisition finished successfully. An access token was retuned:
    Access Token Hash: //removed
    Refresh Token Hash: [No Refresh Token]
    Expiration Time: 04/23/2018 20:04:53 +00:00
    User Hash: null

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54:  - AuthenticationStoreTokenCache: Serializing token cache with 2 items.
[Common.Authentication]: Authenticating using Account: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', environment: 'AzureCloud', tenant: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 

04/23/2018 19:04:54: 1cdd67fa-ce99-4931-b1aa-4ced126fa018 - AcquireTokenForClientHandler: === Token Acquisition started:
    Authority: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/
    Resource: https://datalake.azure.net/
    ClientId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    CacheType: Microsoft.Azure.Commands.Common.Authentication.AuthenticationStoreTokenCache (2 items)
    Authentication Target: Client


Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: Looking up cache for a token...
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - TokenCache: An item matching the requested resource was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Verbose: 1 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: 59.9828124866667 minutes left until token in cache expires
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - <RunAsync>d__0: A matching item (access token or refresh token or both) was found in the cache
Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 :

04/23/2018 19:04:54: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - AcquireTokenHandlerBase: === Token Acquisition finished successfully. An access token was retuned:
    Access Token Hash: //remove
    Refresh Token Hash: [No Refresh Token]
    Expiration Time: 04/23/2018 20:04:53 +00:00
    User Hash: null

Microsoft.IdentityModel.Clients.ActiveDirectory Information: 2 : 04/23/2018 19:04:54:  - AuthenticationStoreTokenCache: Serializing token cache with 2 items.

3 回复 | 直到 8 年前

Joo Wan Ro 8 年前

除了Matt的评论之外,您是否可以尝试通过删除所有模块并仅导入AzureRM来刷新模块。DataLakeAnalytics?正在导入AzureRM。DataLakeAnalytics应自动导入AzureRM。Profile,它是唯一的从属模块。

Mikhail Kholodkov 8 年前

尝试删除(或重命名)所有 newtonsoft。json。动态链接库 您在中拥有的文件 C: \Windows\Assembly\GAC 和 C: \Windows\Assembly\GAC\U MSIL 目录。

Akedren 8 年前

看起来,自从我打开这个问题以来,情况发生了变化,现在我得到了一条不同的(更有用的错误消息)。

Environments                                                                                                            
------------                                                                                                            

{[AzureUSGovernment, AzureUSGovernment], [AzureChinaCloud, AzureChinaCloud], 
[AzureGermanCloud, AzureGermanCloud], [A...
Submit-AzureRmDataLakeAnalyticsJob : The user is not authorized to perform this operation on storage.

At line:21 char:1
+ Submit-AzureRmDataLakeAnalyticsJob -AccountName $adla -Script $script ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : CloseError: (:) [Submit-AzureRmDataLakeAnalyticsJob], CloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.DataLakeAnalytics.SubmitAzureDataLakeAnalyticsJob

我必须添加Azure Automation RunAs帐户才能访问基础Azure Data Lake存储帐户,因为虽然它从订阅的参与者那里继承了对计算的访问权限,但它没有获得Azure Data Lake存储帐户的默认权限。

我是从Azure Data Lake的“添加用户向导”中执行此操作的

我现在看到Azure Automation用户能够在Azure Data Lake Analytics刀片中针对作业管理器触发作业。