Skip to content

Commit 9bc19b7

Browse files
ViktorHofertmdsjkoritzinsky
authored
[release/7.0] source build backports (#78395)
* Use generated runtime.json when building shared framework packages. (#76068) * Use generated runtime.json when building shared framework packages. * Don't UpdateRuntimeJson on Build. * PR feedback. * Update Microsoft.NETCore.Platforms.csproj Co-authored-by: Viktor Hofer <[email protected]> * source-build: support building runtime using non-portable runtime packages. (#75597) * source-build: support building runtime using non-portable runtime packages. Currently source-build performs a 'runtime-portable' build that produces 'linux-{arch}' packages that are used when building target runtime (non-portable). With this change, we can use the non-portable packages that are produced by a previous (non-portable) 'runtime' build. This helps eliminate the 'runtime-portable' build. * Avoid references to Microsoft.AspNetCore.App.<rid>. * crossgen2: set RuntimeIdentifier explicitly. * PR feedback * Revert "crossgen2: set RuntimeIdentifier explicitly." This reverts commit 10be274. * PR feedback. * Revert changes to ILCompiler.csproj. * Update Directory.Build.targets Co-authored-by: Viktor Hofer <[email protected]> * source-build: set the properties to build for mono runtime. (#77786) * Pass TargetRid and SourceBuildNonPortable to the native scripts (#74504) * Pass SourceBuild TargetRid and SourceBuildPortable args through the native script. * Rename -packagerid to -distrorid. * Fix init-distro-rid.sh. * Use OutputRid instead of PackageRid. * Drop unnecessary Condition. * Try passing --distrorid as OutputRid. * NativeExport: use PackageRID for AppHostRuntimeIdentifier. * Rename --distrorid to --outputrid. * Fix SourceBuild.props. * Undo changes to init-distro-rid.sh. * Let source-build leg build 'banana-x64' rid. * SourceBuild.props: don't force a RuntimeOS. * SourceBuild.props: set AdditionalRuntimeIdentifierParent. * Build banana.24-x64 instead. * Fix SourceBuild.props. * SourceBuild.props: try fix AdditionalRuntimeIdentifier* usage. * source-build.yml: add runtimeOS parameter. * SourceBuild.props: derive RuntimeOS from NETCoreSdkRuntimeIdentifier. * Undo using NETCoreSdkRuntimeIdentifier. * Update eng/pipelines/common/global-build-job.yml Co-authored-by: Jeremy Koritzinsky <[email protected]> Co-authored-by: Jeremy Koritzinsky <[email protected]> Co-authored-by: Tom Deseyn <[email protected]> Co-authored-by: Jeremy Koritzinsky <[email protected]>
1 parent 548c2a1 commit 9bc19b7

File tree

16 files changed

+82
-24
lines changed

16 files changed

+82
-24
lines changed

‎Directory.Build.props‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -274,7 +274,6 @@
274274
<PackageProjectUrl>https://dot.net</PackageProjectUrl>
275275
<Owners>microsoft,dotnetframework</Owners>
276276
<IncludeSymbols>true</IncludeSymbols>
277-
<RuntimeIdGraphDefinitionFile>$([MSBuild]::NormalizePath('$(LibrariesProjectRoot)', 'Microsoft.NETCore.Platforms', 'src', 'runtime.json'))</RuntimeIdGraphDefinitionFile>
278277
<LicenseFile>$(MSBuildThisFileDirectory)LICENSE.TXT</LicenseFile>
279278
<PackageLicenseExpression>MIT</PackageLicenseExpression>
280279
<PackageRequireLicenseAcceptance>false</PackageRequireLicenseAcceptance>

‎Directory.Build.targets‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,21 @@
1212
<Import Project="$(RepositoryEngineeringDir)generators.targets" />
1313
<Import Project="$(RepositoryEngineeringDir)python.targets" />
1414

15+
<!--
16+
When .NET gets built from source, make the SDK aware there are bootstrap packages
17+
for Microsoft.NETCore.App.Runtime.<rid> and Microsoft.NETCore.App.Crossgen2.<rid>.
18+
-->
19+
<ItemGroup Condition="'$(DotNetBuildFromSource)' == 'true'">
20+
<KnownFrameworkReference Update="@(KnownFrameworkReference->WithMetadataValue('Identity', 'Microsoft.NETCore.App')->WithMetadataValue('TargetFramework', '$(NetCoreAppCurrent)'))">
21+
<RuntimePackRuntimeIdentifiers>$(PackageRID)</RuntimePackRuntimeIdentifiers>
22+
</KnownFrameworkReference>
23+
<KnownCrossgen2Pack Update="@(KnownCrossgen2Pack->WithMetadataValue('Identity', 'Microsoft.NETCore.App.Crossgen2')->WithMetadataValue('TargetFramework', '$(NetCoreAppCurrent)'))">
24+
<Crossgen2RuntimeIdentifiers>$(PackageRID)</Crossgen2RuntimeIdentifiers>
25+
</KnownCrossgen2Pack>
26+
<!-- Avoid references to Microsoft.AspNetCore.App.Runtime.<rid> -->
27+
<KnownFrameworkReference Remove="Microsoft.AspNetCore.App" />
28+
</ItemGroup>
29+
1530
<PropertyGroup>
1631
<!--
1732
Define this here (not just in Versions.props) because the SDK resets it

‎eng/SourceBuild.props‎

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,36 +11,46 @@
1111
<SourceBuildPortable>true</SourceBuildPortable>
1212
<SourceBuildPortable Condition="'$(SourceBuildNonPortable)' == 'true'">false</SourceBuildPortable>
1313

14-
<!-- If TargetRid not specified, detect RID based on portability. -->
14+
<!-- TargetRid names what gets built. -->
1515
<TargetRid Condition="'$(TargetRid)' == '' and '$(SourceBuildNonPortable)' == 'true'">$([System.Runtime.InteropServices.RuntimeInformation]::RuntimeIdentifier)</TargetRid>
1616
<TargetRid Condition="'$(TargetRid)' == ''">$(__DistroRid)</TargetRid>
1717

1818
<!-- Split e.g. 'fedora.33-x64' into 'fedora.33' and 'x64'. -->
1919
<_targetRidPlatformIndex>$(TargetRid.LastIndexOf('-'))</_targetRidPlatformIndex>
20-
<TargetRidWithoutPlatform>$(TargetRid.Substring(0, $(_targetRidPlatformIndex)))</TargetRidWithoutPlatform>
21-
<TargetRidPlatform>$(TargetRid.Substring($(_targetRidPlatformIndex)).TrimStart('-'))</TargetRidPlatform>
20+
<TargetArch>$(TargetRid.Substring($(_targetRidPlatformIndex)).TrimStart('-'))</TargetArch>
21+
22+
<!-- RuntimeOS is the build host rid OS. -->
23+
<RuntimeOS>$(TargetRid.Substring(0, $(_targetRidPlatformIndex)))</RuntimeOS>
24+
25+
<!-- BaseOS is an expected known rid in the graph that TargetRid is compatible with.
26+
It's used to add TargetRid in the graph if the parent can't be detected. -->
27+
<BaseOS>$(RuntimeOS)</BaseOS>
2228

2329
<LogVerbosity Condition="'$(LogVerbosity)' == ''">minimal</LogVerbosity>
2430
</PropertyGroup>
2531

2632
<Target Name="GetRuntimeSourceBuildCommandConfiguration"
2733
BeforeTargets="GetSourceBuildCommandConfiguration">
2834
<PropertyGroup>
29-
<InnerBuildArgs>$(InnerBuildArgs) --arch $(TargetRidPlatform)</InnerBuildArgs>
35+
<InnerBuildArgs>$(InnerBuildArgs) --arch $(TargetArch)</InnerBuildArgs>
3036
<InnerBuildArgs>$(InnerBuildArgs) --configuration $(Configuration)</InnerBuildArgs>
3137
<InnerBuildArgs Condition="'$(SourceBuildNonPortable)' == 'true'">$(InnerBuildArgs) --allconfigurations</InnerBuildArgs>
3238
<InnerBuildArgs>$(InnerBuildArgs) --verbosity $(LogVerbosity)</InnerBuildArgs>
3339
<InnerBuildArgs>$(InnerBuildArgs) --nodereuse false</InnerBuildArgs>
3440
<InnerBuildArgs>$(InnerBuildArgs) --warnAsError false</InnerBuildArgs>
35-
<InnerBuildArgs>$(InnerBuildArgs) /p:PackageRid=$(TargetRid)</InnerBuildArgs>
41+
<InnerBuildArgs>$(InnerBuildArgs) --outputrid $(TargetRid)</InnerBuildArgs>
42+
<InnerBuildArgs>$(InnerBuildArgs) --portablebuild $(SourceBuildPortable)</InnerBuildArgs>
3643
<InnerBuildArgs>$(InnerBuildArgs) /p:NoPgoOptimize=true</InnerBuildArgs>
3744
<InnerBuildArgs>$(InnerBuildArgs) /p:KeepNativeSymbols=true</InnerBuildArgs>
38-
<InnerBuildArgs>$(InnerBuildArgs) /p:RuntimeOS=$(TargetRidWithoutPlatform)</InnerBuildArgs>
39-
<InnerBuildArgs>$(InnerBuildArgs) /p:PortableBuild=$(SourceBuildPortable)</InnerBuildArgs>
45+
<InnerBuildArgs>$(InnerBuildArgs) /p:RuntimeOS=$(RuntimeOS)</InnerBuildArgs>
46+
<InnerBuildArgs Condition="'$(OfficialBuildId)' != ''">$(InnerBuildArgs) /p:OfficialBuildId=$(OfficialBuildId)</InnerBuildArgs>
47+
<InnerBuildArgs Condition="'$(ContinuousIntegrationBuild)' != ''">$(InnerBuildArgs) /p:ContinuousIntegrationBuild=$(ContinuousIntegrationBuild)</InnerBuildArgs>
4048
<InnerBuildArgs>$(InnerBuildArgs) /p:BuildDebPackage=false</InnerBuildArgs>
4149
<InnerBuildArgs>$(InnerBuildArgs) /p:EnableNgenOptimization=false</InnerBuildArgs>
4250
<InnerBuildArgs>$(InnerBuildArgs) /p:EnablePackageValidation=false</InnerBuildArgs>
4351
<InnerBuildArgs>$(InnerBuildArgs) /p:DisableSourceLink=false</InnerBuildArgs>
52+
<InnerBuildArgs>$(InnerBuildArgs) /p:AdditionalRuntimeIdentifierParent=$(BaseOS)</InnerBuildArgs>
53+
<InnerBuildArgs Condition="'$(SourceBuildUseMonoRuntime)' == 'true'">$(InnerBuildArgs) /p:PrimaryRuntimeFlavor=Mono /p:RuntimeFlavor=Mono</InnerBuildArgs>
4454
</PropertyGroup>
4555
</Target>
4656

‎eng/build.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ usage()
3131
echo " --os Target operating system: windows, Linux, FreeBSD, OSX, MacCatalyst, tvOS,"
3232
echo " tvOSSimulator, iOS, iOSSimulator, Android, Browser, NetBSD, illumos or Solaris."
3333
echo " [Default: Your machine's OS.]"
34+
echo " --outputrid <rid> Optional argument that overrides the target rid name."
3435
echo " --projects <value> Project or solution file(s) to build."
3536
echo " --runtimeConfiguration (-rc) Runtime build configuration: Debug, Release or Checked."
3637
echo " Checked is exclusive to the CLR runtime. It is the same as Debug, except code is"
@@ -402,6 +403,15 @@ while [[ $# > 0 ]]; do
402403
shift 1
403404
;;
404405

406+
-outputrid)
407+
if [ -z ${2+x} ]; then
408+
echo "No value for outputrid is supplied. See help (--help) for supported values." 1>&2
409+
exit 1
410+
fi
411+
arguments="$arguments /p:OutputRid=$(echo "$2" | tr "[:upper:]" "[:lower:]")"
412+
shift 2
413+
;;
414+
405415
-portablebuild)
406416
if [ -z ${2+x} ]; then
407417
echo "No value for portablebuild is supplied. See help (--help) for supported values." 1>&2

‎eng/common/templates/steps/source-build.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,11 @@ steps:
6363
targetRidArgs='/p:TargetRid=${{ parameters.platform.targetRID }}'
6464
fi
6565
66+
runtimeOsArgs=
67+
if [ '${{ parameters.platform.runtimeOS }}' != '' ]; then
68+
runtimeOsArgs='/p:RuntimeOS=${{ parameters.platform.runtimeOS }}'
69+
fi
70+
6671
publishArgs=
6772
if [ '${{ parameters.platform.skipPublishValidation }}' != 'true' ]; then
6873
publishArgs='--publish'
@@ -75,6 +80,7 @@ steps:
7580
$internalRuntimeDownloadArgs \
7681
$internalRestoreArgs \
7782
$targetRidArgs \
83+
$runtimeOsArgs \
7884
/p:SourceBuildNonPortable=${{ parameters.platform.nonPortable }} \
7985
/p:ArcadeBuildFromSource=true
8086
displayName: Build

‎eng/liveBuilds.targets‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,8 @@
222222
ResolveLibrariesRuntimeFilesFromLocalBuild" />
223223

224224
<PropertyGroup>
225-
<BundledRuntimeIdentifierGraphFile>$(RuntimeIdGraphDefinitionFile)</BundledRuntimeIdentifierGraphFile>
225+
<!-- Keep in sync with outputs defined in Microsoft.NETCore.Platforms.csproj. -->
226+
<BundledRuntimeIdentifierGraphFile>$([MSBuild]::NormalizePath('$(ArtifactsBinDir)', 'Microsoft.NETCore.Platforms', 'runtime.json'))</BundledRuntimeIdentifierGraphFile>
227+
<BundledRuntimeIdentifierGraphFile Condition="!Exists('$(BundledRuntimeIdentifierGraphFile)')">$([MSBuild]::NormalizePath('$(LibrariesProjectRoot)', 'Microsoft.NETCore.Platforms', 'src', 'runtime.json'))</BundledRuntimeIdentifierGraphFile>
226228
</PropertyGroup>
227229
</Project>

‎eng/native/build-commons.sh‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -212,6 +212,7 @@ usage()
212212
echo "-gccx.y: optional argument to build using gcc version x.y."
213213
echo "-ninja: target ninja instead of GNU make"
214214
echo "-numproc: set the number of build processes."
215+
echo "-outputrid: optional argument that overrides the target rid name."
215216
echo "-portablebuild: pass -portablebuild=false to force a non-portable build."
216217
echo "-skipconfigure: skip build configuration."
217218
echo "-keepnativesymbols: keep native/unmanaged debug symbols."
@@ -232,6 +233,7 @@ __TargetArch=$arch
232233
__TargetOS=$os
233234
__HostOS=$os
234235
__BuildOS=$os
236+
__OutputRid=''
235237

236238
# Get the number of processors available to the scheduler
237239
# Other techniques such as `nproc` only get the number of
@@ -396,6 +398,16 @@ while :; do
396398
__TargetArch=wasm
397399
;;
398400

401+
outputrid|-outputrid)
402+
if [[ -n "$2" ]]; then
403+
__OutputRid="$2"
404+
shift
405+
else
406+
echo "ERROR: 'outputrid' requires a non-empty option argument"
407+
exit 1
408+
fi
409+
;;
410+
399411
ppc64le|-ppc64le)
400412
__TargetArch=ppc64le
401413
;;
@@ -478,3 +490,7 @@ fi
478490

479491
# init the target distro name
480492
initTargetDistroRid
493+
494+
if [ -z "$__OutputRid" ]; then
495+
__OutputRid="$(echo $__DistroRid | tr '[:upper:]' '[:lower:]')"
496+
fi

‎eng/pipelines/common/global-build-job.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,9 @@ jobs:
141141
platform:
142142
buildScript: $(_sclEnableCommand) $(Build.SourcesDirectory)$(dir)build$(scriptExt)
143143
nonPortable: true
144+
# Use a custom RID that isn't in the RID graph here to validate we don't break the usage of custom rids that aren't in the graph.
145+
targetRID: banana.24-x64
146+
runtimeOS: linux
144147

145148
- ${{ if in(parameters.osGroup, 'OSX', 'iOS', 'tvOS', 'MacCatalyst') }}:
146149
- script: $(Build.SourcesDirectory)/eng/install-native-dependencies.sh ${{ parameters.osGroup }} ${{ parameters.archType }} azDO

‎src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<Project Sdk="Microsoft.NET.Sdk">
22
<PropertyGroup>
33
<OutputPath>$(RuntimeBinDir)ilc/</OutputPath>
4-
<RuntimeIdentifier>$(OutputRid)</RuntimeIdentifier>
4+
<RuntimeIdentifier>$(PackageRID)</RuntimeIdentifier>
55
</PropertyGroup>
66

77
<Import Project="ILCompiler.props" />

‎src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717
<AvoidRestoreCycleOnSelfReference>true</AvoidRestoreCycleOnSelfReference>
1818
<!-- TODO: Remove with AvoidRestoreCycleOnSelfReference hack. -->
1919
<PackageValidationBaselineName>$(MSBuildProjectName)</PackageValidationBaselineName>
20-
<BeforePack>GenerateRuntimeJson;UpdateRuntimeJson;$(BeforePack)</BeforePack>
2120

2221
<_generateRuntimeGraphTargetFramework Condition="'$(MSBuildRuntimeType)' == 'core'">$(NetCoreAppToolCurrent)</_generateRuntimeGraphTargetFramework>
2322
<_generateRuntimeGraphTargetFramework Condition="'$(MSBuildRuntimeType)' != 'core'">net472</_generateRuntimeGraphTargetFramework>
@@ -44,7 +43,7 @@
4443

4544
<ItemGroup>
4645
<Content Condition="'$(AdditionalRuntimeIdentifiers)' == ''" Include="runtime.json" PackagePath="/" />
47-
<Content Condition="'$(AdditionalRuntimeIdentifiers)' != ''" Include="$(IntermediateOutputPath)runtime.json" PackagePath="/" />
46+
<Content Condition="'$(AdditionalRuntimeIdentifiers)' != ''" Include="$(BaseOutputPath)runtime.json" PackagePath="/" />
4847
<Content Include="$(PlaceholderFile)" PackagePath="lib/netstandard1.0" />
4948
</ItemGroup>
5049

@@ -58,12 +57,12 @@
5857

5958
<UsingTask TaskName="GenerateRuntimeGraph" AssemblyFile="$(_generateRuntimeGraphTask)"/>
6059

61-
<Target Name="GenerateRuntimeJson" Condition="'$(AdditionalRuntimeIdentifiers)' != ''">
60+
<Target Name="GenerateRuntimeJson" AfterTargets="Build" Condition="'$(AdditionalRuntimeIdentifiers)' != ''">
6261
<MakeDir Directories="$(IntermediateOutputPath)" />
6362
<GenerateRuntimeGraph RuntimeGroups="@(RuntimeGroupWithQualifiers)"
6463
AdditionalRuntimeIdentifiers="$(AdditionalRuntimeIdentifiers)"
6564
AdditionalRuntimeIdentifierParent="$(AdditionalRuntimeIdentifierParent)"
66-
RuntimeJson="$(IntermediateOutputPath)runtime.json"
65+
RuntimeJson="$(BaseOutputPath)runtime.json"
6766
UpdateRuntimeFiles="True" />
6867
</Target>
6968

0 commit comments

Comments
 (0)