Skip to content

DXIL.dll crashes on intrinsics that are not defined for a known shader model #6168

@jenatali

Description

@jenatali

Description
Given DXIL that declares itself as shader model 6.7 / DXIL 1.7, but uses intrinsic functions defined by shader model 6.8, attempting to validate that DXIL module will crash with an access violation.

Steps to Reproduce
See the relevant code in dxil::OP:

void OP::RefreshCache() {
  for (Function &F : m_pModule->functions()) {
    if (OP::IsDxilOpFunc(&F) && !F.user_empty()) {
      CallInst *CI = cast<CallInst>(*F.user_begin());
      OpCode OpCode = OP::GetDxilOpFuncCallInst(CI);
      Type *pOverloadType = OP::GetOverloadType(OpCode, &F);
      Function *OpFunc = GetOpFunc(OpCode, pOverloadType);
      (void)(OpFunc);
      DXASSERT_NOMSG(OpFunc == &F);
    }
  }
}

For every intrinsic function, GetOpFunc is called to instantiate it in the cache. Inside GetOpFunc:

Function *OP::GetOpFunc(OpCode opCode, Type *pOverloadType) {
  DXASSERT(0 <= (unsigned)opCode && opCode < OpCode::NumOpCodes,
           "otherwise caller passed OOB OpCode");
  assert(0 <= (unsigned)opCode && opCode < OpCode::NumOpCodes);
  DXASSERT(IsOverloadLegal(opCode, pOverloadType),
           "otherwise the caller requested illegal operation overload (eg HLSL "
           "function with unsupported types for mapped intrinsic function)");
  OpCodeClass opClass = m_OpCodeProps[(unsigned)opCode].opCodeClass;
  Function *&F =
      m_OpCodeClassCache[(unsigned)opClass].pOverloads[pOverloadType];

Those asserts for "otherwise caller..." should be validation, because the caller clearly didn't do any validation. As a result, indexing into m_OpCodeProps can use an out-of-bounds index, meaning that accessing opCodeClass could be an access violation directly, or at best could return garbage. I was seeing it return a pointer-type value, which was then used to index into m_OpCodeClassCache, which failed catastrophically.

My repro was done by accidentally inserting SampleCmpBias instructions into a SM6.7 shader in the Mesa DXIL backend. DXC will not emit such a shader, since it has validation against doing so earlier on in the compilation process.

Metadata

Metadata

Assignees

Labels

bugBug, regression, crashcrashDXC crashing or hitting an assertsm6.8Shader Model 6.8validationRelated to validation or signing

Type

No type

Projects

Status

Done

Status

Triaged

Relationships

None yet

Development

No branches or pull requests

Issue actions