Jul 14 15:46:42.897069 ip-10-0-139-147 systemd[1]: Starting Kubernetes Kubelet... Jul 14 15:46:43.311968 ip-10-0-139-147 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 15:46:43.311968 ip-10-0-139-147 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 15:46:43.311968 ip-10-0-139-147 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 15:46:43.311968 ip-10-0-139-147 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 15:46:43.311968 ip-10-0-139-147 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 15:46:43.313001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.312862 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 15:46:43.321034 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321011 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 15:46:43.321034 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321030 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 15:46:43.321034 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321035 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 15:46:43.321034 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321040 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 15:46:43.321034 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321044 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321049 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321053 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321057 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321064 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321069 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321073 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321078 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321083 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321088 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321092 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321096 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321100 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321112 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321117 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321121 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321125 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321130 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321133 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321138 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 15:46:43.321329 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321142 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321147 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321151 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321156 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321160 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321164 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321168 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321172 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321176 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321180 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321185 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321189 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321193 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321198 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321203 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321210 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321215 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321220 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321225 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 15:46:43.322192 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321230 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321236 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321240 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321245 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321249 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321253 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321257 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321262 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321266 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321271 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321276 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321280 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321285 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321289 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321293 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321298 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321303 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321307 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321311 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 15:46:43.323017 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321316 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321320 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321324 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321328 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321334 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321339 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321343 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321347 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321353 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321358 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321363 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321367 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321373 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321377 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321381 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321386 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321390 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321394 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321399 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321406 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 15:46:43.323832 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321412 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321416 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321423 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321428 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321433 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321437 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321441 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321447 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321451 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321455 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321460 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321464 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321469 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321474 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321478 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321483 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.321487 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322672 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322682 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322687 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 15:46:43.324737 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322694 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322698 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322704 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322709 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322714 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322718 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322723 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322727 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322732 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322736 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322740 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322745 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322749 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322754 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322758 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322763 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322767 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322772 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322776 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322780 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 15:46:43.325600 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322784 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322789 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322793 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322797 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322827 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322831 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322836 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322840 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322844 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322848 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322852 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322856 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322862 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322867 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322874 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322882 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322887 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322892 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322897 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322901 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 15:46:43.326343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322906 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322910 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322915 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322919 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322924 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322930 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322936 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322941 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322947 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322951 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322955 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322960 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322964 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322969 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322972 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322977 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322981 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322985 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322990 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322994 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 15:46:43.326945 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.322999 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323003 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323007 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323012 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323018 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323023 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323027 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323032 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323038 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323042 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323047 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323053 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323059 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323064 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323070 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323074 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323079 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323084 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323088 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 15:46:43.327427 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323095 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323101 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323105 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323110 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323115 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323119 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323124 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323128 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323132 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323136 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323141 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323145 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323150 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323154 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323159 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323163 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323167 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323861 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323872 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323877 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 15:46:43.328161 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323882 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323893 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323898 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323903 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323907 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323911 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323916 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323921 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323926 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323930 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323935 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323939 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323944 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323949 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323953 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323957 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323961 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323966 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323970 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323974 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 15:46:43.328666 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323978 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323983 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323987 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323991 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323995 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.323999 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324003 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324008 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324012 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324017 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324022 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324026 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324031 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324035 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324039 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324046 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324050 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324055 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324060 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324064 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 15:46:43.329299 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324068 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324072 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324076 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324080 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324085 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324090 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324095 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324099 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324104 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324109 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324113 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324117 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324121 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324125 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324130 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324134 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324139 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324143 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324147 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324151 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 15:46:43.329796 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324156 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324162 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324168 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324173 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324178 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324183 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324188 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324200 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324205 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324209 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324214 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324218 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324222 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324226 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324230 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324234 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324242 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324248 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324253 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 15:46:43.330365 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324259 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324265 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324270 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324275 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324280 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324285 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324289 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324293 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324298 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324302 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324306 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324310 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324315 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324319 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324323 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324328 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.324332 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324433 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324445 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324460 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324467 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 15:46:43.330850 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324490 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324501 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324515 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324528 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324533 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324539 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324545 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324550 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324554 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324559 2573 flags.go:64] FLAG: --cgroup-root="" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324563 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324568 2573 flags.go:64] FLAG: --client-ca-file="" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324573 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324577 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324588 2573 flags.go:64] FLAG: --cluster-domain="" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324593 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324599 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324604 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324625 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324632 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324637 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324643 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324648 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324654 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 14 15:46:43.331361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324664 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324669 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324675 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324680 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324687 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324692 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324697 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324702 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324707 2573 flags.go:64] FLAG: --enable-server="true" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324721 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324737 2573 flags.go:64] FLAG: --event-burst="100" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324742 2573 flags.go:64] FLAG: --event-qps="50" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324747 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324752 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324757 2573 flags.go:64] FLAG: --eviction-hard="" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324772 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324778 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324783 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324788 2573 flags.go:64] FLAG: --eviction-soft="" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324793 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324818 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324824 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324829 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324834 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324838 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 15:46:43.331990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324843 2573 flags.go:64] FLAG: --feature-gates="" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324850 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324856 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324861 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324866 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324871 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324877 2573 flags.go:64] FLAG: --help="false" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324884 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324889 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324894 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324899 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324905 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324911 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324915 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324921 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324926 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324931 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324937 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324942 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324947 2573 flags.go:64] FLAG: --kube-reserved="" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324952 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324957 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324963 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324969 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 15:46:43.332856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324974 2573 flags.go:64] FLAG: --lock-file="" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324979 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324984 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324989 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.324999 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325006 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325011 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325016 2573 flags.go:64] FLAG: --logging-format="text" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325021 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325027 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325031 2573 flags.go:64] FLAG: --manifest-url="" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325036 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325045 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325050 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325057 2573 flags.go:64] FLAG: --max-pods="110" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325064 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325069 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325073 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325078 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325084 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325089 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325094 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325105 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325110 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325115 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 15:46:43.333908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325120 2573 flags.go:64] FLAG: --pod-cidr="" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325127 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325135 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325140 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325146 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325151 2573 flags.go:64] FLAG: --port="10250" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325156 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325162 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ca35e73301bac150" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325167 2573 flags.go:64] FLAG: --qos-reserved="" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325173 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325178 2573 flags.go:64] FLAG: --register-node="true" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325183 2573 flags.go:64] FLAG: --register-with-taints="" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325189 2573 flags.go:64] FLAG: --registry-burst="10" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325194 2573 flags.go:64] FLAG: --registry-qps="5" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325198 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325203 2573 flags.go:64] FLAG: --reserved-memory="" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325210 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325216 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325221 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325226 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325230 2573 flags.go:64] FLAG: --runonce="false" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325235 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325243 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325248 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325253 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 15:46:43.334996 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325258 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325264 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325269 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325274 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325279 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325284 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325289 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325294 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325299 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325305 2573 flags.go:64] FLAG: --system-cgroups="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325310 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325319 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325325 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325330 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325338 2573 flags.go:64] FLAG: --tls-min-version="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325343 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325348 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325353 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325358 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325362 2573 flags.go:64] FLAG: --v="2" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325369 2573 flags.go:64] FLAG: --version="false" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325375 2573 flags.go:64] FLAG: --vmodule="" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325382 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.325388 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325581 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 15:46:43.336174 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325588 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325592 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325597 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325602 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325608 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325612 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325617 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325621 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325625 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325629 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325634 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325638 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325642 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325647 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325652 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325656 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325660 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325669 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325675 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325680 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 15:46:43.336934 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325685 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325691 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325696 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325700 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325705 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325709 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325713 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325718 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325725 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325731 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325736 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325740 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325745 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325749 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325754 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325758 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325764 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325769 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325773 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 15:46:43.337788 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325777 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325781 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325786 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325790 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325794 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325819 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325824 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325829 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325833 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325837 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325842 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325848 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325853 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325857 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325863 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325868 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325873 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325884 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325889 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 15:46:43.338551 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325894 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325898 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325903 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325908 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325913 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325917 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325922 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325926 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325931 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325935 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325942 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325946 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325950 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325954 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325959 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325963 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325968 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325973 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325977 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325982 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 15:46:43.339065 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325986 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325990 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325995 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.325999 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326003 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326009 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326013 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326019 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326024 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326029 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326033 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326037 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326043 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326047 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326052 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326056 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326062 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326069 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326073 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 15:46:43.339562 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.326078 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.326704 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.334747 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.334767 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.334786 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.334796 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334897 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334904 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334912 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334918 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334923 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334928 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334933 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334938 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 15:46:43.340048 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334943 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334948 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334952 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334958 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334962 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334967 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334971 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334977 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334981 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334985 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334990 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334994 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.334998 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335002 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335006 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335010 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335015 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335019 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335022 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335027 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 15:46:43.340436 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335031 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335037 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335041 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335046 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335050 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335055 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335059 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335063 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335068 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335072 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335076 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335081 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335087 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335093 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335098 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335103 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335107 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335112 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335116 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 15:46:43.340940 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335121 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335125 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335129 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335134 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335138 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335142 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335146 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335151 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335155 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335160 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335164 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335168 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335175 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335180 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335186 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335193 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335198 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335202 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335206 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335211 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 15:46:43.341408 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335215 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335219 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335224 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335228 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335232 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335237 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335241 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335245 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335249 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335254 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335259 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335263 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335268 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335272 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335278 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335282 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335286 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335290 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335295 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335299 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 15:46:43.341914 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335304 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335309 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335313 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335317 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335322 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335326 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335330 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335334 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335340 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335345 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335349 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335354 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.335362 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335541 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335549 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 15:46:43.342405 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335555 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335560 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335565 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335570 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335575 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335580 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335585 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335590 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335595 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335600 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335605 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335609 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335614 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335619 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335623 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335628 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335632 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335637 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335641 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 15:46:43.342776 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335645 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335649 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335654 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335658 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335662 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335666 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335671 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335676 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335680 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335685 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335689 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335693 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335698 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335702 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335706 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335710 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335714 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335719 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335724 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335728 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 15:46:43.343256 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335732 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335736 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335740 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335745 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335749 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335754 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335761 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335766 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335771 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335776 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335781 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335785 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335789 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335794 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335818 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335823 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335827 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335830 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335835 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 15:46:43.343750 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335839 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335844 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335848 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335852 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335857 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335861 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335865 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335868 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335872 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335876 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335880 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335884 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335889 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335893 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335897 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335902 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335907 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335912 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335916 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335920 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 15:46:43.344317 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335925 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335929 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335933 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335938 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335943 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335948 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335952 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335959 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335965 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335971 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335975 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335981 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335985 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335990 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.335999 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.336004 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.336009 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.336013 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 15:46:43.344825 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:43.336017 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.336025 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.336185 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.339877 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.341062 2573 server.go:1019] "Starting client certificate rotation" Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.341161 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.341197 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.343576 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 15:46:43.345266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.344793 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 15:46:43.354547 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.354525 2573 log.go:25] "Validated CRI v1 runtime API" Jul 14 15:46:43.357369 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.357354 2573 log.go:25] "Validated CRI v1 image API" Jul 14 15:46:43.358725 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.358709 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 15:46:43.360960 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.360941 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9ae88fa0-3ce7-4b59-82df-93040e255547:/dev/nvme0n1p4 edb500c8-2b48-4f5b-b6ae-ea050b11fefe:/dev/nvme0n1p3] Jul 14 15:46:43.361032 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.360959 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 15:46:43.366541 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.366429 2573 manager.go:216] Machine: {Timestamp:2026-07-14 15:46:43.365183746 +0000 UTC m=+0.342509054 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099960 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d688f0261837ff7a63b6e817d726e SystemUUID:ec2d688f-0261-837f-f7a6-3b6e817d726e BootID:3cdd0cc9-336e-4620-9b3b-fe32f837d9ed Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:09:e2:69:12:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:09:e2:69:12:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:54:37:4f:15:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 15:46:43.366541 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.366538 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 15:46:43.366662 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.366650 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 15:46:43.368207 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.368193 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 15:46:43.369666 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.369640 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 15:46:43.369812 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.369671 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-147.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 15:46:43.370413 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.370401 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 15:46:43.370453 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.370417 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 15:46:43.370453 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.370425 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 15:46:43.371561 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.371550 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 15:46:43.373375 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.373365 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 15:46:43.376219 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376208 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 14 15:46:43.376514 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376504 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 15:46:43.376544 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376537 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 14 15:46:43.376570 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376548 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 15:46:43.376570 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376559 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 15:46:43.376570 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376567 2573 kubelet.go:398] "Adding apiserver pod source" Jul 14 15:46:43.376669 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.376576 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 15:46:43.377093 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.377078 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 15:46:43.377761 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.377747 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 15:46:43.380902 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.380652 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 15:46:43.382683 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382669 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 15:46:43.382739 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382700 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 15:46:43.382903 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382893 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 15:46:43.382936 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382908 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 15:46:43.382936 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382914 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 15:46:43.382936 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382919 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 15:46:43.382936 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382926 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 15:46:43.382936 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.382932 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 15:46:43.383303 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.383294 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 15:46:43.383333 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.383304 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 15:46:43.383333 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.383311 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 15:46:43.383333 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.383316 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 15:46:43.383333 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.383324 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 15:46:43.383333 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.383332 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 15:46:43.384052 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.384043 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 15:46:43.384052 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.384052 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 15:46:43.387577 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.387562 2573 server.go:1268] "Started kubelet" Jul 14 15:46:43.387693 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.387643 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 15:46:43.387749 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.387741 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 15:46:43.388123 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.387913 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 15:46:43.388551 ip-10-0-139-147 systemd[1]: Started Kubernetes Kubelet. Jul 14 15:46:43.389209 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.388998 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 15:46:43.389296 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.389277 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 14 15:46:43.390317 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.390284 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-147.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 15:46:43.390408 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.390316 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 15:46:43.390408 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.390337 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-147.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 15:46:43.393069 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.392157 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-147.ec2.internal.18c232919d947872 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-147.ec2.internal,UID:ip-10-0-139-147.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-147.ec2.internal,},FirstTimestamp:2026-07-14 15:46:43.387537522 +0000 UTC m=+0.364862831,LastTimestamp:2026-07-14 15:46:43.387537522 +0000 UTC m=+0.364862831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-147.ec2.internal,}" Jul 14 15:46:43.393744 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.393720 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 15:46:43.395476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.395452 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 15:46:43.395588 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.395564 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.395588 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.395459 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 15:46:43.396114 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396093 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 15:46:43.396215 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396166 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 15:46:43.396215 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396183 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 15:46:43.396340 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396324 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 14 15:46:43.396340 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396336 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 14 15:46:43.396479 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396404 2573 factory.go:55] Registering systemd factory Jul 14 15:46:43.396479 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396419 2573 factory.go:223] Registration of the systemd container factory successfully Jul 14 15:46:43.396689 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396674 2573 factory.go:153] Registering CRI-O factory Jul 14 15:46:43.396747 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396694 2573 factory.go:223] Registration of the crio container factory successfully Jul 14 15:46:43.396747 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396734 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 15:46:43.396883 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396754 2573 factory.go:103] Registering Raw factory Jul 14 15:46:43.396883 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.396767 2573 manager.go:1196] Started watching for new ooms in manager Jul 14 15:46:43.397224 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.397210 2573 manager.go:349] Starting recovery of all containers Jul 14 15:46:43.397992 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.397970 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d2g6c" Jul 14 15:46:43.399047 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.399024 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-147.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 15:46:43.399302 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.399246 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 15:46:43.403840 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.403790 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d2g6c" Jul 14 15:46:43.407969 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.407946 2573 manager.go:354] Recovery completed Jul 14 15:46:43.413306 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.413293 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.415364 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.415347 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.415427 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.415379 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.415427 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.415396 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.416013 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.415996 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 15:46:43.416081 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.416011 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.416081 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.416014 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 15:46:43.416081 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.416056 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 14 15:46:43.416766 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.416747 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.416870 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.416777 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.416870 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.416788 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.417249 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.417238 2573 policy_none.go:49] "None policy: Start" Jul 14 15:46:43.417282 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.417254 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 15:46:43.417282 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.417264 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 15:46:43.417840 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.417728 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-147.ec2.internal.18c232919f3d1358 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-147.ec2.internal,UID:ip-10-0-139-147.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-147.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-147.ec2.internal,},FirstTimestamp:2026-07-14 15:46:43.41536444 +0000 UTC m=+0.392689752,LastTimestamp:2026-07-14 15:46:43.41536444 +0000 UTC m=+0.392689752,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-147.ec2.internal,}" Jul 14 15:46:43.418286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.418274 2573 policy_none.go:47] "Start" Jul 14 15:46:43.460477 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.460450 2573 manager.go:337] "Starting Device Plugin manager" Jul 14 15:46:43.460591 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.460492 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 15:46:43.460591 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.460506 2573 server.go:87] "Starting device plugin registration server" Jul 14 15:46:43.460825 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.460792 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 15:46:43.460902 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.460828 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 15:46:43.460954 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.460921 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 15:46:43.461010 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.460996 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 15:46:43.461061 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.461011 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 15:46:43.461518 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.461493 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 15:46:43.461610 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.461547 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.499712 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.499669 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 15:46:43.500960 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.500942 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 15:46:43.501049 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.500988 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 15:46:43.501049 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.501012 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 15:46:43.501146 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.501048 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 15:46:43.503260 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.503240 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:43.561233 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.561201 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.562974 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.562922 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.562974 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.562958 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.562974 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.562969 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.563126 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.562993 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.569904 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.569888 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.570006 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.569909 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-147.ec2.internal\": node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.588186 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.588154 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.601165 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.601121 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal"] Jul 14 15:46:43.601284 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.601199 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.602149 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.602133 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.602216 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.602162 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.602216 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.602172 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.603511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.603498 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.603647 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.603628 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.603709 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.603689 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.605623 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.605605 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.605724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.605635 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.605724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.605646 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.605724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.605611 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.605724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.605709 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.605724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.605725 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.606950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.606935 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.606998 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.606961 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 15:46:43.607731 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.607714 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientMemory" Jul 14 15:46:43.607843 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.607747 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 15:46:43.607843 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.607760 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeHasSufficientPID" Jul 14 15:46:43.620392 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.620371 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-147.ec2.internal\" not found" node="ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.624344 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.624329 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-147.ec2.internal\" not found" node="ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.688399 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.688369 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.697675 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.697647 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/02df77f8e782e2edef6cb31070a11871-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal\" (UID: \"02df77f8e782e2edef6cb31070a11871\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.697738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.697677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02df77f8e782e2edef6cb31070a11871-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal\" (UID: \"02df77f8e782e2edef6cb31070a11871\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.697738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.697696 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fa1b573b1bff3bbe3f49d2310b1a627-config\") pod \"kube-apiserver-proxy-ip-10-0-139-147.ec2.internal\" (UID: \"9fa1b573b1bff3bbe3f49d2310b1a627\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.788697 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.788668 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.798070 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.798049 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/02df77f8e782e2edef6cb31070a11871-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal\" (UID: \"02df77f8e782e2edef6cb31070a11871\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.798131 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.798078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02df77f8e782e2edef6cb31070a11871-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal\" (UID: \"02df77f8e782e2edef6cb31070a11871\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.798131 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.798096 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fa1b573b1bff3bbe3f49d2310b1a627-config\") pod \"kube-apiserver-proxy-ip-10-0-139-147.ec2.internal\" (UID: \"9fa1b573b1bff3bbe3f49d2310b1a627\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.798207 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.798141 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/02df77f8e782e2edef6cb31070a11871-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal\" (UID: \"02df77f8e782e2edef6cb31070a11871\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.798207 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.798153 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fa1b573b1bff3bbe3f49d2310b1a627-config\") pod \"kube-apiserver-proxy-ip-10-0-139-147.ec2.internal\" (UID: \"9fa1b573b1bff3bbe3f49d2310b1a627\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.798207 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.798145 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02df77f8e782e2edef6cb31070a11871-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal\" (UID: \"02df77f8e782e2edef6cb31070a11871\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.889555 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.889474 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:43.921967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.921942 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.927250 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:43.927235 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" Jul 14 15:46:43.990092 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:43.990049 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:45.681895 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:45.681848 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:45.782517 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:45.782428 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:45.804439 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.804414 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:45.883266 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:45.883230 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-147.ec2.internal\" not found" Jul 14 15:46:45.900265 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.900239 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:45.932390 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.932367 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 15:46:45.932942 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.932489 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 15:46:45.932942 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.932531 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 15:46:45.932942 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.932531 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 15:46:45.968698 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.968668 2573 apiserver.go:52] "Watching apiserver" Jul 14 15:46:45.986634 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.986608 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 15:46:45.987355 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.987330 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bg8f6","kube-system/konnectivity-agent-gwrkq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd","openshift-cluster-node-tuning-operator/tuned-kmdr8","openshift-image-registry/node-ca-hpv8z","openshift-multus/multus-additional-cni-plugins-hklg2","openshift-multus/network-metrics-daemon-l5rh8","openshift-network-diagnostics/network-check-target-8l64k","openshift-dns/node-resolver-g6vvg","openshift-multus/multus-xcmxd","openshift-network-operator/iptables-alerter-mwvp9","openshift-ovn-kubernetes/ovnkube-node-bffbw"] Jul 14 15:46:45.987355 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.987350 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 15:46:45.987530 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.987358 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" Jul 14 15:46:45.988759 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.988741 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:45.988847 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:45.988797 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:45.989730 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.989714 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:45.991204 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.991175 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:45.991542 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.991523 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rnnwp\"" Jul 14 15:46:45.992031 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.992011 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 15:46:45.992130 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.992078 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 15:46:45.992476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.992458 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:45.993186 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.993166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-kubelet-config\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:45.993273 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.993202 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-dbus\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:45.993273 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.993231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:45.993372 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.993304 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 15:46:45.993731 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.993709 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:45.994230 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.994209 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ddl8k\"" Jul 14 15:46:45.994316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.994207 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 15:46:45.994316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.994277 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 15:46:45.994504 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.994485 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 15:46:45.994571 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.994557 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 15:46:45.994682 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.994634 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jlpxs\"" Jul 14 15:46:45.995343 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.995327 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:45.996333 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.996317 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 15:46:45.996532 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.996516 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:45.996602 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:45.996583 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:45.996887 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.996872 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 15:46:45.997084 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997067 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 15:46:45.997143 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997077 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r6fxk\"" Jul 14 15:46:45.997403 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997387 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 15:46:45.997475 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997403 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 15:46:45.997661 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997646 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:45.997698 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997661 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 15:46:45.997735 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997703 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 15:46:45.997735 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:45.997705 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:45.997873 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.997858 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s8qgb\"" Jul 14 15:46:45.998138 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.998123 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 15:46:45.998910 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.998895 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:45.998950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.998901 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 15:41:44 +0000 UTC" deadline="2028-02-25 15:00:52.498008558 +0000 UTC" Jul 14 15:46:45.998950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.998937 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14183h14m6.499075233s" Jul 14 15:46:45.999275 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.999261 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 15:46:45.999326 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:45.999321 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" Jul 14 15:46:46.000194 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.000180 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.000884 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.000868 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 15:46:46.001118 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.001094 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9sd4n\"" Jul 14 15:46:46.001184 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.001164 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 15:46:46.001350 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.001336 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.002214 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.002200 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 15:46:46.002396 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.002380 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 15:46:46.002486 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.002470 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal"] Jul 14 15:46:46.002559 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.002547 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.003148 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.003133 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5dl98\"" Jul 14 15:46:46.003468 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.003455 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 15:46:46.003575 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.003562 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 15:46:46.003652 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.003564 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 15:46:46.003705 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.003670 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-btfwv\"" Jul 14 15:46:46.004356 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.004340 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 15:46:46.004431 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.004360 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 15:46:46.004822 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.004808 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 15:46:46.005011 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.004995 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 15:46:46.005093 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.005065 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 15:46:46.005159 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.005137 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 15:46:46.005248 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.005232 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwqcx\"" Jul 14 15:46:46.011005 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.010986 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal"] Jul 14 15:46:46.011005 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.011000 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 15:46:46.023521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.023502 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pdh8k" Jul 14 15:46:46.031648 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.031632 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pdh8k" Jul 14 15:46:46.088427 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.088359 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 15:46:46.093642 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093615 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-cni-binary-copy\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.093734 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-sys-fs\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.093734 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093686 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-modprobe-d\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.093734 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-sys\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.093857 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t8xj\" (UniqueName: \"kubernetes.io/projected/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-kube-api-access-4t8xj\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.093857 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:46.093946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r69gp\" (UniqueName: \"kubernetes.io/projected/c581dd53-95b4-46a5-bd65-87722abac294-kube-api-access-r69gp\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.093946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093877 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-cni-multus\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.093946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093912 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-k8s-cni-cncf-io\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.093946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093942 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-registration-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-device-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.093989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-hosts-file\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094013 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovnkube-config\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094029 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-env-overrides\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094118 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-cnibin\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094141 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8252700-8c60-4a8b-9906-65b993067113-multus-daemon-config\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094158 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-cni-bin\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094176 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysconfig\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094192 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30d24472-389d-4270-93b1-4495ddd241bb-tmp\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-os-release\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094259 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-system-cni-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-netns\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094308 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-cni-bin\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094344 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-log-socket\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-run\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094395 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxjpb\" (UniqueName: \"kubernetes.io/projected/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-kube-api-access-zxjpb\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094427 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-kubelet-config\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-cni-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094476 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-os-release\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094498 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-kubelet\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094521 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlwnr\" (UniqueName: \"kubernetes.io/projected/e8252700-8c60-4a8b-9906-65b993067113-kube-api-access-hlwnr\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.094521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094527 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-kubelet-config\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094543 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-run-netns\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-var-lib-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-socket-dir-parent\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094609 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-etc-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-ovn\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094668 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-cni-netd\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-socket-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094711 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-systemd\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094732 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-lib-modules\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094839 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094874 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8252700-8c60-4a8b-9906-65b993067113-cni-binary-copy\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.094883 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094915 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysctl-d\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.094935 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:46.594916044 +0000 UTC m=+1.981003410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:46.095167 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.094993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-host\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw7kv\" (UniqueName: \"kubernetes.io/projected/d96c9827-c29c-4426-927a-e9710682b720-kube-api-access-nw7kv\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095076 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-kubelet\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-systemd\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095136 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-run-ovn-kubernetes\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovn-node-metrics-cert\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095197 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-var-lib-kubelet\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095212 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xmpp\" (UniqueName: \"kubernetes.io/projected/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-kube-api-access-8xmpp\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-hostroot\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aaab5b07-517a-4305-a812-7e84dabe36fc-iptables-alerter-script\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f4ln\" (UniqueName: \"kubernetes.io/projected/aaab5b07-517a-4305-a812-7e84dabe36fc-kube-api-access-8f4ln\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095294 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxvjk\" (UniqueName: \"kubernetes.io/projected/61c359d7-8c6e-413e-a068-8981fbd29cf6-kube-api-access-hxvjk\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095316 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-systemd-units\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095331 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095360 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovnkube-script-lib\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.095783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095392 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-system-cni-dir\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095406 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-conf-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-slash\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095444 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-node-log\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095488 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-host\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095501 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-tmp-dir\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095530 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-dbus\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095552 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095567 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-cnibin\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30d24472-389d-4270-93b1-4495ddd241bb-etc-tuned\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095613 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phnlt\" (UniqueName: \"kubernetes.io/projected/30d24472-389d-4270-93b1-4495ddd241bb-kube-api-access-phnlt\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095627 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-serviceca\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8720cae8-3d43-472c-a510-7b8d5667da1d-agent-certs\") pod \"konnectivity-agent-gwrkq\" (UID: \"8720cae8-3d43-472c-a510-7b8d5667da1d\") " pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095683 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-dbus\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:46.096483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095690 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.097001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095715 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-multus-certs\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.097001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095733 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-etc-kubernetes\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.097001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-kubernetes\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.097001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095762 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysctl-conf\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.097001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095778 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaab5b07-517a-4305-a812-7e84dabe36fc-host-slash\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.097001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.095794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8720cae8-3d43-472c-a510-7b8d5667da1d-konnectivity-ca\") pod \"konnectivity-agent-gwrkq\" (UID: \"8720cae8-3d43-472c-a510-7b8d5667da1d\") " pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.196361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-netns\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-cni-bin\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-log-socket\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-run\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196412 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxjpb\" (UniqueName: \"kubernetes.io/projected/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-kube-api-access-zxjpb\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-cni-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196471 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-netns\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196486 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-log-socket\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196470 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-cni-bin\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196500 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-run\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-os-release\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196527 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-os-release\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-kubelet\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlwnr\" (UniqueName: \"kubernetes.io/projected/e8252700-8c60-4a8b-9906-65b993067113-kube-api-access-hlwnr\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-run-netns\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196592 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-kubelet\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-var-lib-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196548 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-cni-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196640 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-run-netns\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196649 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-var-lib-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196712 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-socket-dir-parent\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196740 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-etc-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-ovn\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196788 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-socket-dir-parent\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-cni-netd\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196819 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-socket-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196791 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-etc-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196835 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-ovn\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-systemd\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.196908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-lib-modules\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196884 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-systemd\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196835 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-cni-netd\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196900 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8252700-8c60-4a8b-9906-65b993067113-cni-binary-copy\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.196933 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-socket-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197009 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-lib-modules\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysctl-d\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197410 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysctl-d\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197518 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-host\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw7kv\" (UniqueName: \"kubernetes.io/projected/d96c9827-c29c-4426-927a-e9710682b720-kube-api-access-nw7kv\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-kubelet\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.197692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197666 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-kubelet\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197734 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-systemd\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197824 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8252700-8c60-4a8b-9906-65b993067113-cni-binary-copy\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197854 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197863 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-host\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197899 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-openvswitch\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197911 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-run-systemd\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197917 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-run-ovn-kubernetes\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.197973 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-run-ovn-kubernetes\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovn-node-metrics-cert\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-var-lib-kubelet\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198123 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xmpp\" (UniqueName: \"kubernetes.io/projected/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-kube-api-access-8xmpp\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-hostroot\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aaab5b07-517a-4305-a812-7e84dabe36fc-iptables-alerter-script\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.198225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198212 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-var-lib-kubelet\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.198741 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198526 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-hostroot\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.198741 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198665 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f4ln\" (UniqueName: \"kubernetes.io/projected/aaab5b07-517a-4305-a812-7e84dabe36fc-kube-api-access-8f4ln\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.198843 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198819 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxvjk\" (UniqueName: \"kubernetes.io/projected/61c359d7-8c6e-413e-a068-8981fbd29cf6-kube-api-access-hxvjk\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.198891 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198856 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-systemd-units\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198936 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198894 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.198985 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198946 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 15:46:46.198985 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.198975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovnkube-script-lib\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.199100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199005 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.199100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199057 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-system-cni-dir\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.199100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-conf-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.199231 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-slash\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.199231 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-node-log\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.199320 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.199320 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199274 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-host\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.199320 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199304 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-tmp-dir\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.199448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.199448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-cnibin\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.199448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199389 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30d24472-389d-4270-93b1-4495ddd241bb-etc-tuned\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.199448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199415 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phnlt\" (UniqueName: \"kubernetes.io/projected/30d24472-389d-4270-93b1-4495ddd241bb-kube-api-access-phnlt\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.199448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199445 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-serviceca\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.199660 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8720cae8-3d43-472c-a510-7b8d5667da1d-agent-certs\") pod \"konnectivity-agent-gwrkq\" (UID: \"8720cae8-3d43-472c-a510-7b8d5667da1d\") " pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199885 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-multus-certs\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199888 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-system-cni-dir\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-etc-kubernetes\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199954 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-kubernetes\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.199986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysctl-conf\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaab5b07-517a-4305-a812-7e84dabe36fc-host-slash\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200025 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-slash\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200107 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200114 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200144 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aaab5b07-517a-4305-a812-7e84dabe36fc-host-slash\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200163 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-multus-conf-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200217 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-node-log\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200271 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200314 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-host\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.199987 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:46.200560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200421 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8720cae8-3d43-472c-a510-7b8d5667da1d-konnectivity-ca\") pod \"konnectivity-agent-gwrkq\" (UID: \"8720cae8-3d43-472c-a510-7b8d5667da1d\") " pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200598 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-systemd-units\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200780 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200841 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-cnibin\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.200976 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysctl-conf\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201205 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-tmp-dir\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201256 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovnkube-script-lib\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201262 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201312 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-serviceca\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201367 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-multus-certs\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.201410 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:46.701385384 +0000 UTC m=+2.087472764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:46.201487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201446 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-cni-binary-copy\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.202011 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201527 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-kubernetes\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.202011 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-sys-fs\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.202011 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201854 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aaab5b07-517a-4305-a812-7e84dabe36fc-iptables-alerter-script\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.202198 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.202174 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c581dd53-95b4-46a5-bd65-87722abac294-cni-binary-copy\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.202252 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.202240 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8720cae8-3d43-472c-a510-7b8d5667da1d-konnectivity-ca\") pod \"konnectivity-agent-gwrkq\" (UID: \"8720cae8-3d43-472c-a510-7b8d5667da1d\") " pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.202606 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.202528 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovn-node-metrics-cert\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.202606 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.202548 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-etc-kubernetes\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.202794 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.201950 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-sys-fs\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.202794 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.202727 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-modprobe-d\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.203256 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203174 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-modprobe-d\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.203256 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-sys\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.203412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t8xj\" (UniqueName: \"kubernetes.io/projected/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-kube-api-access-4t8xj\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.203412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:46.203412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203324 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r69gp\" (UniqueName: \"kubernetes.io/projected/c581dd53-95b4-46a5-bd65-87722abac294-kube-api-access-r69gp\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.203412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-cni-multus\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.203412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-k8s-cni-cncf-io\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.203412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203408 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-registration-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.203709 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203539 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-device-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.203709 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203603 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-sys\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203439 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-device-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-hosts-file\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovnkube-config\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-env-overrides\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.203930 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-cnibin\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204085 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-hosts-file\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204178 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61c359d7-8c6e-413e-a068-8981fbd29cf6-registration-dir\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204343 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-var-lib-cni-multus\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-host-run-k8s-cni-cncf-io\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.204703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204587 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8720cae8-3d43-472c-a510-7b8d5667da1d-agent-certs\") pod \"konnectivity-agent-gwrkq\" (UID: \"8720cae8-3d43-472c-a510-7b8d5667da1d\") " pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204719 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-cnibin\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204732 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-ovnkube-config\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204785 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8252700-8c60-4a8b-9906-65b993067113-multus-daemon-config\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204820 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-env-overrides\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204940 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-cni-bin\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204960 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-host-cni-bin\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.205222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.204998 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysconfig\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.205608 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205288 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/30d24472-389d-4270-93b1-4495ddd241bb-etc-sysconfig\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.205608 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30d24472-389d-4270-93b1-4495ddd241bb-tmp\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.205608 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205416 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-os-release\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.205608 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-system-cni-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.205608 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205516 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8252700-8c60-4a8b-9906-65b993067113-multus-daemon-config\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.205608 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205539 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8252700-8c60-4a8b-9906-65b993067113-system-cni-dir\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.205871 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205681 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/30d24472-389d-4270-93b1-4495ddd241bb-etc-tuned\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.205871 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205690 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:46.205871 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.205770 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c581dd53-95b4-46a5-bd65-87722abac294-os-release\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.207796 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.207763 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxjpb\" (UniqueName: \"kubernetes.io/projected/00d6d8d2-df0c-4093-a8c0-201d3ef36d37-kube-api-access-zxjpb\") pod \"ovnkube-node-bffbw\" (UID: \"00d6d8d2-df0c-4093-a8c0-201d3ef36d37\") " pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.208395 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.208330 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw7kv\" (UniqueName: \"kubernetes.io/projected/d96c9827-c29c-4426-927a-e9710682b720-kube-api-access-nw7kv\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:46.208492 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.208438 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xmpp\" (UniqueName: \"kubernetes.io/projected/bd82b629-05a6-4aa2-9a72-1c421ae37e0d-kube-api-access-8xmpp\") pod \"node-resolver-g6vvg\" (UID: \"bd82b629-05a6-4aa2-9a72-1c421ae37e0d\") " pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.208649 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.208622 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlwnr\" (UniqueName: \"kubernetes.io/projected/e8252700-8c60-4a8b-9906-65b993067113-kube-api-access-hlwnr\") pod \"multus-xcmxd\" (UID: \"e8252700-8c60-4a8b-9906-65b993067113\") " pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.210337 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.209305 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:46:46.210337 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.209330 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:46:46.210337 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.209343 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:46.210337 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.209396 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:46:46.709380047 +0000 UTC m=+2.095467425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:46.210337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.209951 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f4ln\" (UniqueName: \"kubernetes.io/projected/aaab5b07-517a-4305-a812-7e84dabe36fc-kube-api-access-8f4ln\") pod \"iptables-alerter-mwvp9\" (UID: \"aaab5b07-517a-4305-a812-7e84dabe36fc\") " pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.210337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.210284 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phnlt\" (UniqueName: \"kubernetes.io/projected/30d24472-389d-4270-93b1-4495ddd241bb-kube-api-access-phnlt\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.210695 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.210561 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30d24472-389d-4270-93b1-4495ddd241bb-tmp\") pod \"tuned-kmdr8\" (UID: \"30d24472-389d-4270-93b1-4495ddd241bb\") " pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.210828 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.210806 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxvjk\" (UniqueName: \"kubernetes.io/projected/61c359d7-8c6e-413e-a068-8981fbd29cf6-kube-api-access-hxvjk\") pod \"aws-ebs-csi-driver-node-l8jtd\" (UID: \"61c359d7-8c6e-413e-a068-8981fbd29cf6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.211979 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.211823 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r69gp\" (UniqueName: \"kubernetes.io/projected/c581dd53-95b4-46a5-bd65-87722abac294-kube-api-access-r69gp\") pod \"multus-additional-cni-plugins-hklg2\" (UID: \"c581dd53-95b4-46a5-bd65-87722abac294\") " pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.212739 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.212721 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t8xj\" (UniqueName: \"kubernetes.io/projected/ae7ada8b-e8d8-489d-bf68-b3b820771c1a-kube-api-access-4t8xj\") pod \"node-ca-hpv8z\" (UID: \"ae7ada8b-e8d8-489d-bf68-b3b820771c1a\") " pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.213641 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.213624 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 15:46:46.312659 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.312559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:46:46.321498 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.321465 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8720cae8_3d43_472c_a510_7b8d5667da1d.slice/crio-6d4040523c2f87d2e27f4b151db596a3f1f4cca4c496a3a47711f091fbcf1c97 WatchSource:0}: Error finding container 6d4040523c2f87d2e27f4b151db596a3f1f4cca4c496a3a47711f091fbcf1c97: Status 404 returned error can't find the container with id 6d4040523c2f87d2e27f4b151db596a3f1f4cca4c496a3a47711f091fbcf1c97 Jul 14 15:46:46.343506 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.343488 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" Jul 14 15:46:46.349511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.349481 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" Jul 14 15:46:46.351105 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.351081 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c359d7_8c6e_413e_a068_8981fbd29cf6.slice/crio-073b84197eea1afcad3b700bde25d4ffe7bb445604009a40d555b1046ea76f1b WatchSource:0}: Error finding container 073b84197eea1afcad3b700bde25d4ffe7bb445604009a40d555b1046ea76f1b: Status 404 returned error can't find the container with id 073b84197eea1afcad3b700bde25d4ffe7bb445604009a40d555b1046ea76f1b Jul 14 15:46:46.356826 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.356802 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d24472_389d_4270_93b1_4495ddd241bb.slice/crio-2b5718c75612426457c4aac456ab035238b143c17d8137609d171ddc7d3f231c WatchSource:0}: Error finding container 2b5718c75612426457c4aac456ab035238b143c17d8137609d171ddc7d3f231c: Status 404 returned error can't find the container with id 2b5718c75612426457c4aac456ab035238b143c17d8137609d171ddc7d3f231c Jul 14 15:46:46.383955 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.383921 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa1b573b1bff3bbe3f49d2310b1a627.slice/crio-79f051782e64bc83ac431bd0a075d5470ec1e19488458d72b50165305b135343 WatchSource:0}: Error finding container 79f051782e64bc83ac431bd0a075d5470ec1e19488458d72b50165305b135343: Status 404 returned error can't find the container with id 79f051782e64bc83ac431bd0a075d5470ec1e19488458d72b50165305b135343 Jul 14 15:46:46.384464 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.384444 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hpv8z" Jul 14 15:46:46.393416 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.393395 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7ada8b_e8d8_489d_bf68_b3b820771c1a.slice/crio-eaaab248af5d26d3d9032b21bc01740c75512bba8cb5300cf5d156eded7608ef WatchSource:0}: Error finding container eaaab248af5d26d3d9032b21bc01740c75512bba8cb5300cf5d156eded7608ef: Status 404 returned error can't find the container with id eaaab248af5d26d3d9032b21bc01740c75512bba8cb5300cf5d156eded7608ef Jul 14 15:46:46.400274 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.400254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hklg2" Jul 14 15:46:46.407428 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.407406 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc581dd53_95b4_46a5_bd65_87722abac294.slice/crio-4c93c89505d05cd67c4b5b9e7a3c32abdbe574ff75a793ddf4c43f77762a5957 WatchSource:0}: Error finding container 4c93c89505d05cd67c4b5b9e7a3c32abdbe574ff75a793ddf4c43f77762a5957: Status 404 returned error can't find the container with id 4c93c89505d05cd67c4b5b9e7a3c32abdbe574ff75a793ddf4c43f77762a5957 Jul 14 15:46:46.420641 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.420614 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g6vvg" Jul 14 15:46:46.428117 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.428091 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd82b629_05a6_4aa2_9a72_1c421ae37e0d.slice/crio-b7e7b0905f14405c9a15a0036465c00ce7b4afc3e42f98d3ea12bbb9f7e3cb51 WatchSource:0}: Error finding container b7e7b0905f14405c9a15a0036465c00ce7b4afc3e42f98d3ea12bbb9f7e3cb51: Status 404 returned error can't find the container with id b7e7b0905f14405c9a15a0036465c00ce7b4afc3e42f98d3ea12bbb9f7e3cb51 Jul 14 15:46:46.443510 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.443492 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xcmxd" Jul 14 15:46:46.450381 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.450358 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8252700_8c60_4a8b_9906_65b993067113.slice/crio-956ba5a343768a959839241fd7bccc17330dde31676d6ccd6a7efa727242bf07 WatchSource:0}: Error finding container 956ba5a343768a959839241fd7bccc17330dde31676d6ccd6a7efa727242bf07: Status 404 returned error can't find the container with id 956ba5a343768a959839241fd7bccc17330dde31676d6ccd6a7efa727242bf07 Jul 14 15:46:46.458517 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.458500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mwvp9" Jul 14 15:46:46.463685 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.463622 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:46:46.465852 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.465829 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaab5b07_517a_4305_a812_7e84dabe36fc.slice/crio-4a4c1dd945394194f97805eef1a5b6f1d99d52a31e2d8b05e1e05cb5704a111f WatchSource:0}: Error finding container 4a4c1dd945394194f97805eef1a5b6f1d99d52a31e2d8b05e1e05cb5704a111f: Status 404 returned error can't find the container with id 4a4c1dd945394194f97805eef1a5b6f1d99d52a31e2d8b05e1e05cb5704a111f Jul 14 15:46:46.470964 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:46:46.470943 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d6d8d2_df0c_4093_a8c0_201d3ef36d37.slice/crio-3a02d79367475658fc510ea7d53e1fbd128222b0a5ec66adfb2b53be7232761c WatchSource:0}: Error finding container 3a02d79367475658fc510ea7d53e1fbd128222b0a5ec66adfb2b53be7232761c: Status 404 returned error can't find the container with id 3a02d79367475658fc510ea7d53e1fbd128222b0a5ec66adfb2b53be7232761c Jul 14 15:46:46.608318 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.608208 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:46.608455 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.608354 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:46.608455 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.608412 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:47.608397406 +0000 UTC m=+2.994484774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:46.709192 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.709166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:46.709355 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.709263 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:46.709355 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.709312 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:47.709298572 +0000 UTC m=+3.095385937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:46.810346 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.810313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:46.810515 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.810432 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:46:46.810515 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.810446 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:46:46.810515 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.810455 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:46.810515 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:46.810502 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:46:47.810489111 +0000 UTC m=+3.196576476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:46.847016 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:46.846984 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:47.034019 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.033898 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 15:41:46 +0000 UTC" deadline="2028-03-14 05:06:54.093404627 +0000 UTC" Jul 14 15:46:47.034019 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.033970 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14605h20m7.059439069s" Jul 14 15:46:47.094903 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.094868 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:47.095089 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.095002 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:47.112173 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.112074 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xcmxd" event={"ID":"e8252700-8c60-4a8b-9906-65b993067113","Type":"ContainerStarted","Data":"956ba5a343768a959839241fd7bccc17330dde31676d6ccd6a7efa727242bf07"} Jul 14 15:46:47.122217 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.122178 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerStarted","Data":"4c93c89505d05cd67c4b5b9e7a3c32abdbe574ff75a793ddf4c43f77762a5957"} Jul 14 15:46:47.129856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.129814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hpv8z" event={"ID":"ae7ada8b-e8d8-489d-bf68-b3b820771c1a","Type":"ContainerStarted","Data":"eaaab248af5d26d3d9032b21bc01740c75512bba8cb5300cf5d156eded7608ef"} Jul 14 15:46:47.136729 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.136694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" event={"ID":"9fa1b573b1bff3bbe3f49d2310b1a627","Type":"ContainerStarted","Data":"79f051782e64bc83ac431bd0a075d5470ec1e19488458d72b50165305b135343"} Jul 14 15:46:47.151460 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.148471 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" event={"ID":"30d24472-389d-4270-93b1-4495ddd241bb","Type":"ContainerStarted","Data":"2b5718c75612426457c4aac456ab035238b143c17d8137609d171ddc7d3f231c"} Jul 14 15:46:47.156240 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.156205 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"3a02d79367475658fc510ea7d53e1fbd128222b0a5ec66adfb2b53be7232761c"} Jul 14 15:46:47.165552 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.165519 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g6vvg" event={"ID":"bd82b629-05a6-4aa2-9a72-1c421ae37e0d","Type":"ContainerStarted","Data":"b7e7b0905f14405c9a15a0036465c00ce7b4afc3e42f98d3ea12bbb9f7e3cb51"} Jul 14 15:46:47.170219 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.170188 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" event={"ID":"61c359d7-8c6e-413e-a068-8981fbd29cf6","Type":"ContainerStarted","Data":"073b84197eea1afcad3b700bde25d4ffe7bb445604009a40d555b1046ea76f1b"} Jul 14 15:46:47.184746 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.184712 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gwrkq" event={"ID":"8720cae8-3d43-472c-a510-7b8d5667da1d","Type":"ContainerStarted","Data":"6d4040523c2f87d2e27f4b151db596a3f1f4cca4c496a3a47711f091fbcf1c97"} Jul 14 15:46:47.192902 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.192869 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" event={"ID":"02df77f8e782e2edef6cb31070a11871","Type":"ContainerStarted","Data":"306af0b500295cd12f3387d2c1b058f585c840166388772147e6f920c8982d34"} Jul 14 15:46:47.203968 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.203884 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mwvp9" event={"ID":"aaab5b07-517a-4305-a812-7e84dabe36fc","Type":"ContainerStarted","Data":"4a4c1dd945394194f97805eef1a5b6f1d99d52a31e2d8b05e1e05cb5704a111f"} Jul 14 15:46:47.617053 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.616993 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:47.617254 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.617205 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:47.617329 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.617283 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:49.617252267 +0000 UTC m=+5.003339637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:47.718466 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.718431 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:47.718649 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.718588 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:47.718708 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.718650 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:49.71863333 +0000 UTC m=+5.104720710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:47.819696 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.819656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:47.819880 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.819849 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:46:47.819880 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.819869 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:46:47.819986 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.819883 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:47.819986 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:47.819944 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:46:49.819925961 +0000 UTC m=+5.206013350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:47.834170 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:47.834115 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:48.034534 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:48.034487 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 15:41:46 +0000 UTC" deadline="2028-01-21 17:21:14.956887494 +0000 UTC" Jul 14 15:46:48.034534 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:48.034532 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13345h34m26.922360904s" Jul 14 15:46:48.094230 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:48.092988 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:48.094230 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:48.093145 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:48.094230 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:48.093552 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:48.094230 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:48.093657 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:49.033455 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:49.033258 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 15:46:49.094707 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:49.094675 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:49.095154 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.094789 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:49.638944 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:49.638906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:49.639163 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.639069 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:49.639163 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.639135 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:53.639114714 +0000 UTC m=+9.025202088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:49.740311 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:49.740273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:49.740487 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.740421 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:49.740545 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.740506 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:53.740468213 +0000 UTC m=+9.126555580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:49.842172 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:49.841410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:49.842172 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.841690 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:46:49.842172 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.841716 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:46:49.842172 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.841728 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:49.842172 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:49.841790 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:46:53.841770426 +0000 UTC m=+9.227857807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:50.092687 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:50.092602 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:50.092687 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:50.092641 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:50.092906 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:50.092747 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:50.092906 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:50.092880 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:51.093251 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:51.093212 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:51.093679 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:51.093354 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:52.093304 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:52.093237 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:52.093304 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:52.093264 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:52.093930 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:52.093378 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:52.093930 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:52.093531 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:53.093363 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:53.093329 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:53.093735 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.093479 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:53.674690 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:53.674648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:53.674877 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.674829 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:53.674941 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.674900 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:01.674881891 +0000 UTC m=+17.060969274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:46:53.776213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:53.776167 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:53.776358 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.776303 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:53.776358 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.776354 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:01.776339032 +0000 UTC m=+17.162426410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:46:53.876575 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:53.876529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:53.876798 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.876745 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:46:53.876798 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.876766 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:46:53.876798 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.876779 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:53.876914 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:53.876841 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:47:01.87682208 +0000 UTC m=+17.262909451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:46:54.093257 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:54.093165 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:54.093563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:54.093542 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:54.093930 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:54.093627 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:54.093930 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:54.093320 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:55.093922 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:55.093519 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:55.093922 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:55.093632 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:56.092952 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:56.092910 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:56.093166 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:56.093051 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:56.093166 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:56.093074 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:56.093302 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:56.093226 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:57.093474 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:57.093433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:57.093972 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:57.093581 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:46:58.092793 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:58.092754 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:46:58.092982 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:58.092901 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:46:58.092982 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:58.092924 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:46:58.093099 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:58.093071 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:46:59.094027 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:46:59.093986 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:46:59.094417 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:46:59.094123 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:00.093012 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:00.092968 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:00.093239 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:00.093159 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:00.093757 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:00.093619 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:00.093757 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:00.093713 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:01.093296 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:01.093258 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:01.093694 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.093387 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:01.729584 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:01.729537 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:01.729780 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.729700 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:47:01.729780 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.729764 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:17.729748543 +0000 UTC m=+33.115835912 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:47:01.830414 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:01.830377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:01.830637 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.830533 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:47:01.830637 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.830606 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:17.830589203 +0000 UTC m=+33.216676569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:47:01.931121 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:01.931082 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:01.931284 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.931245 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:47:01.931284 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.931264 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:47:01.931284 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.931274 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:47:01.931398 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:01.931325 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:47:17.93130929 +0000 UTC m=+33.317396674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:47:02.092641 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:02.092559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:02.092641 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:02.092598 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:02.092823 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:02.092689 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:02.092823 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:02.092809 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:03.093392 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:03.093294 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:03.093875 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:03.093444 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:04.092488 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:04.092453 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:04.092488 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:04.092495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:04.092762 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:04.092601 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:04.092762 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:04.092699 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:05.093938 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:05.093895 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:05.094354 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:05.094015 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:06.093244 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:06.093210 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:06.093446 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:06.093325 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:06.093446 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:06.093385 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:06.093554 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:06.093508 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:07.093797 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:07.093769 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:07.094226 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:07.093900 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:07.260478 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:07.260224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" event={"ID":"9fa1b573b1bff3bbe3f49d2310b1a627","Type":"ContainerStarted","Data":"b86919c3d78c5fab04ab12530378b9a31b638cdc3a48b635a7e0000080eee084"} Jul 14 15:47:07.278873 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:07.278820 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-147.ec2.internal" podStartSLOduration=21.278802515 podStartE2EDuration="21.278802515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:07.278610128 +0000 UTC m=+22.664697516" watchObservedRunningTime="2026-07-14 15:47:07.278802515 +0000 UTC m=+22.664889903" Jul 14 15:47:08.093316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.093075 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:08.093462 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.093164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:08.093462 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:08.093352 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:08.093543 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:08.093479 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:08.263637 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.263599 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" event={"ID":"61c359d7-8c6e-413e-a068-8981fbd29cf6","Type":"ContainerStarted","Data":"3281c78e10d887ec1309ede1abad44f62229cc843b25356c047f20fc0326dd04"} Jul 14 15:47:08.265104 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.265072 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gwrkq" event={"ID":"8720cae8-3d43-472c-a510-7b8d5667da1d","Type":"ContainerStarted","Data":"4d93706ab8defc8dca5e3846aff4cfa33443fcfdb799fb63372dcc97374a0445"} Jul 14 15:47:08.266543 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.266520 2573 generic.go:356] "Generic (PLEG): container finished" podID="02df77f8e782e2edef6cb31070a11871" containerID="4c030a9e0588fe4686b14c7c932ef5c38ad0934c6db3dbf79cb1b481b1bc7d3d" exitCode=0 Jul 14 15:47:08.266629 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.266601 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" event={"ID":"02df77f8e782e2edef6cb31070a11871","Type":"ContainerDied","Data":"4c030a9e0588fe4686b14c7c932ef5c38ad0934c6db3dbf79cb1b481b1bc7d3d"} Jul 14 15:47:08.267904 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.267879 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xcmxd" event={"ID":"e8252700-8c60-4a8b-9906-65b993067113","Type":"ContainerStarted","Data":"13353227c34e7bd3ee37d1b3688097f47572c030dd92971d24c9179141d0134a"} Jul 14 15:47:08.269355 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.269333 2573 generic.go:356] "Generic (PLEG): container finished" podID="c581dd53-95b4-46a5-bd65-87722abac294" containerID="92c75e9c0cc8ea86746ac8cff483a591a7914a180fa4cd750350512a2282ff57" exitCode=0 Jul 14 15:47:08.269449 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.269410 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerDied","Data":"92c75e9c0cc8ea86746ac8cff483a591a7914a180fa4cd750350512a2282ff57"} Jul 14 15:47:08.270865 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.270845 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hpv8z" event={"ID":"ae7ada8b-e8d8-489d-bf68-b3b820771c1a","Type":"ContainerStarted","Data":"4668048669fa8668e10afa6e0b5c67236cb344c579574c4034f0954193a5037c"} Jul 14 15:47:08.272316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.272291 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" event={"ID":"30d24472-389d-4270-93b1-4495ddd241bb","Type":"ContainerStarted","Data":"24c2253301fb3a483fb35b648733257e12ede20f1974d749643bf92eab69046b"} Jul 14 15:47:08.274990 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.274963 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:47:08.275310 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275290 2573 generic.go:356] "Generic (PLEG): container finished" podID="00d6d8d2-df0c-4093-a8c0-201d3ef36d37" containerID="c9c6096c7355281126d7551c6358f076c54b4e49a4ff6c9de68173141fd64e3f" exitCode=1 Jul 14 15:47:08.275377 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"0f3d1c967d70f48fa1b39a05d28590cfc6bb394e7b370c6383078424304d7b63"} Jul 14 15:47:08.275438 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275426 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"6af41d87c5d5abd8548ad2dc639dd396d7a74be9be3563986b11f2d366458ad5"} Jul 14 15:47:08.275494 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"dd59fe0964254eff5b0f2fbfdafc853d716df5e60df4710b646ec03a74f4dafe"} Jul 14 15:47:08.275547 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275496 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"119a8de29ab7b2d6882a8c6842a6cc14080041eabddd3722b6712b1673ee5ac1"} Jul 14 15:47:08.275547 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275525 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerDied","Data":"c9c6096c7355281126d7551c6358f076c54b4e49a4ff6c9de68173141fd64e3f"} Jul 14 15:47:08.275635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.275561 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"ce9f8906150d6476fc271cb4089fb2dbe469fd838870fb0eb3be32e6ed7f7a1b"} Jul 14 15:47:08.276685 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.276667 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g6vvg" event={"ID":"bd82b629-05a6-4aa2-9a72-1c421ae37e0d","Type":"ContainerStarted","Data":"1fce72a56ad5c6d63302222dc2340befd6b4f97e8f830aa9b405647ab221eeb8"} Jul 14 15:47:08.281681 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.281647 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gwrkq" podStartSLOduration=2.533530595 podStartE2EDuration="23.281635815s" totalImagesPullingTime="20.74810522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:08.281283792 +0000 UTC m=+23.667371179" watchObservedRunningTime="2026-07-14 15:47:08.281635815 +0000 UTC m=+23.667723202" Jul 14 15:47:08.295946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.295910 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g6vvg" podStartSLOduration=2.641870564 podStartE2EDuration="23.295898078s" totalImagesPullingTime="20.654027514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:08.29589638 +0000 UTC m=+23.681983768" watchObservedRunningTime="2026-07-14 15:47:08.295898078 +0000 UTC m=+23.681985465" Jul 14 15:47:08.330835 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.330785 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xcmxd" podStartSLOduration=2.12204987 podStartE2EDuration="23.330767577s" totalImagesPullingTime="21.208717707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:08.313759898 +0000 UTC m=+23.699847284" watchObservedRunningTime="2026-07-14 15:47:08.330767577 +0000 UTC m=+23.716854965" Jul 14 15:47:08.331210 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.331187 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kmdr8" podStartSLOduration=2.590438406 podStartE2EDuration="23.331179906s" totalImagesPullingTime="20.7407415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:08.330417426 +0000 UTC m=+23.716504813" watchObservedRunningTime="2026-07-14 15:47:08.331179906 +0000 UTC m=+23.717267293" Jul 14 15:47:08.375523 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:08.375432 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hpv8z" podStartSLOduration=2.6795307680000002 podStartE2EDuration="23.375417355s" totalImagesPullingTime="20.695886587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:08.346085714 +0000 UTC m=+23.732173101" watchObservedRunningTime="2026-07-14 15:47:08.375417355 +0000 UTC m=+23.761504777" Jul 14 15:47:09.093193 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:09.093157 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:09.093395 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:09.093285 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:09.280929 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:09.280888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mwvp9" event={"ID":"aaab5b07-517a-4305-a812-7e84dabe36fc","Type":"ContainerStarted","Data":"fe0bfe6b31bffec8ebdc9d09beab924b384589442f589d811ac66b16153776a2"} Jul 14 15:47:09.298678 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:09.298632 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mwvp9" podStartSLOduration=3.668013879 podStartE2EDuration="24.298616071s" totalImagesPullingTime="20.630602192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:09.298489204 +0000 UTC m=+24.684576591" watchObservedRunningTime="2026-07-14 15:47:09.298616071 +0000 UTC m=+24.684703461" Jul 14 15:47:10.093156 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.092936 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:10.093371 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.092997 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:10.093371 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:10.093268 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:10.093371 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:10.093325 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:10.225858 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.225826 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 15:47:10.287566 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.287538 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:47:10.288111 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.287971 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"223cc3dfbb6d953c0f47a118f36c4c0addd173486ba8de7e21df09b4e9171b46"} Jul 14 15:47:10.290196 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.290165 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" event={"ID":"61c359d7-8c6e-413e-a068-8981fbd29cf6","Type":"ContainerStarted","Data":"2d62b9518822023ad23ba582a864aa836fd51100a10faac27213bdf0c9f3d6b8"} Jul 14 15:47:10.292842 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.292806 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" event={"ID":"02df77f8e782e2edef6cb31070a11871","Type":"ContainerStarted","Data":"75e24fd11dd1aa5b4dcdcbf0f789b05152f9c7ec77c437b3caacd423ba72fae8"} Jul 14 15:47:10.314288 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:10.314183 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-147.ec2.internal" podStartSLOduration=25.314162975 podStartE2EDuration="25.314162975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:10.313265897 +0000 UTC m=+25.699353285" watchObservedRunningTime="2026-07-14 15:47:10.314162975 +0000 UTC m=+25.700250364" Jul 14 15:47:11.072380 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:11.072245 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T15:47:10.225842054Z","UUID":"219bfd99-0e27-4af5-865a-711f4ced9be9","Handler":null,"Name":"","Endpoint":""} Jul 14 15:47:11.073932 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:11.073906 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 15:47:11.074094 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:11.073962 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 15:47:11.092565 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:11.092539 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:11.092737 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:11.092653 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:12.092744 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:12.092707 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:12.093580 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:12.092833 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:12.093580 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:12.092889 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:12.093580 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:12.093009 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:12.591137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:12.591026 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:47:12.591696 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:12.591674 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:47:13.092855 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.092676 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:13.093282 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:13.092943 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:13.301186 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.301088 2573 generic.go:356] "Generic (PLEG): container finished" podID="c581dd53-95b4-46a5-bd65-87722abac294" containerID="52947523427e4986c224e1dcf93037ed0813047297115579c4fc85e57f544415" exitCode=0 Jul 14 15:47:13.301186 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.301170 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerDied","Data":"52947523427e4986c224e1dcf93037ed0813047297115579c4fc85e57f544415"} Jul 14 15:47:13.304963 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.304945 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:47:13.305411 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.305391 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"b0e94705891ed9c1888cf2d8838300a2e9470187dfdc3f32c70e3c246454dbcc"} Jul 14 15:47:13.305643 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.305624 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:47:13.305736 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.305690 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:47:13.305829 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.305814 2573 scope.go:117] "RemoveContainer" containerID="c9c6096c7355281126d7551c6358f076c54b4e49a4ff6c9de68173141fd64e3f" Jul 14 15:47:13.308448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.308412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" event={"ID":"61c359d7-8c6e-413e-a068-8981fbd29cf6","Type":"ContainerStarted","Data":"5de8165709d10f021e11d3e86b6a6cf73a4aadc2addd8ac356d0a50d88707a0f"} Jul 14 15:47:13.308681 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.308664 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:47:13.309206 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.309192 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gwrkq" Jul 14 15:47:13.323161 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.323139 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:47:13.397867 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:13.397820 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8jtd" podStartSLOduration=2.931131075 podStartE2EDuration="28.397800543s" totalImagesPullingTime="25.466669468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:13.364570751 +0000 UTC m=+28.750658137" watchObservedRunningTime="2026-07-14 15:47:13.397800543 +0000 UTC m=+28.783887931" Jul 14 15:47:14.093118 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.092825 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:14.093611 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.092831 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:14.093611 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:14.093478 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:14.098420 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:14.098386 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:14.315092 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.315060 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:47:14.315521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.315495 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" event={"ID":"00d6d8d2-df0c-4093-a8c0-201d3ef36d37","Type":"ContainerStarted","Data":"9c2d3e74a45e7c4c18ceaccb584cea43f881191ecf7a232e95ae06b4e2ebe240"} Jul 14 15:47:14.315837 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.315811 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:47:14.331918 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.331894 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:47:14.412119 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:14.412005 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" podStartSLOduration=8.577919135 podStartE2EDuration="29.411987311s" totalImagesPullingTime="20.834068176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:14.364074731 +0000 UTC m=+29.750162118" watchObservedRunningTime="2026-07-14 15:47:14.411987311 +0000 UTC m=+29.798074697" Jul 14 15:47:15.093240 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.093210 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:15.094030 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:15.093300 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:15.194801 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.194773 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bg8f6"] Jul 14 15:47:15.194962 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.194913 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:15.195019 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:15.194999 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:15.195348 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.195324 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8l64k"] Jul 14 15:47:15.198840 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.198818 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l5rh8"] Jul 14 15:47:15.198963 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.198909 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:15.199031 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:15.199014 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:15.319792 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.319708 2573 generic.go:356] "Generic (PLEG): container finished" podID="c581dd53-95b4-46a5-bd65-87722abac294" containerID="fb337d87022e5dc8d2b7604fe12d31fcc070ef1daad36c93128af086d83271d1" exitCode=0 Jul 14 15:47:15.319953 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.319827 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerDied","Data":"fb337d87022e5dc8d2b7604fe12d31fcc070ef1daad36c93128af086d83271d1"} Jul 14 15:47:15.319953 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:15.319869 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:15.320033 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:15.319996 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:17.092971 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.092775 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:17.093370 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.092778 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:17.093370 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.093086 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:17.093370 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.092830 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:17.093370 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.093137 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:17.093370 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.093237 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:17.326692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.326597 2573 generic.go:356] "Generic (PLEG): container finished" podID="c581dd53-95b4-46a5-bd65-87722abac294" containerID="131a520d141a45a5718abc4cc5e735d8cccbfccf47b5401e5208c4015e317d45" exitCode=0 Jul 14 15:47:17.326692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.326669 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerDied","Data":"131a520d141a45a5718abc4cc5e735d8cccbfccf47b5401e5208c4015e317d45"} Jul 14 15:47:17.746410 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.746319 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:17.746576 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.746468 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 15:47:17.746576 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.746527 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret podName:1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:49.746511351 +0000 UTC m=+65.132598718 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret") pod "global-pull-secret-syncer-bg8f6" (UID: "1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8") : object "kube-system"/"original-pull-secret" not registered Jul 14 15:47:17.847337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.847305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:17.847516 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.847443 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:47:17.847516 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.847507 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs podName:d96c9827-c29c-4426-927a-e9710682b720 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:49.847493263 +0000 UTC m=+65.233580628 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs") pod "network-metrics-daemon-l5rh8" (UID: "d96c9827-c29c-4426-927a-e9710682b720") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 15:47:17.947741 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:17.947700 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:17.947922 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.947854 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 15:47:17.947922 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.947882 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 15:47:17.947922 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.947894 2573 projected.go:196] Error preparing data for projected volume kube-api-access-dbnx9 for pod openshift-network-diagnostics/network-check-target-8l64k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:47:17.948112 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:17.947959 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9 podName:14573d3c-2ea9-4b67-8494-1981ea25e12b nodeName:}" failed. No retries permitted until 2026-07-14 15:47:49.947938699 +0000 UTC m=+65.334026084 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbnx9" (UniqueName: "kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9") pod "network-check-target-8l64k" (UID: "14573d3c-2ea9-4b67-8494-1981ea25e12b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 15:47:19.093211 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:19.093166 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:19.093640 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:19.093243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:19.093640 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:19.093262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:19.093640 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:19.093379 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg8f6" podUID="1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8" Jul 14 15:47:19.093640 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:19.093446 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5rh8" podUID="d96c9827-c29c-4426-927a-e9710682b720" Jul 14 15:47:19.093640 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:19.093521 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8l64k" podUID="14573d3c-2ea9-4b67-8494-1981ea25e12b" Jul 14 15:47:20.446034 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.445998 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-147.ec2.internal" event="NodeReady" Jul 14 15:47:20.446557 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.446166 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 15:47:20.488384 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.488352 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77cf6fc8b9-89rdg"] Jul 14 15:47:20.522850 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.522822 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d7652"] Jul 14 15:47:20.523010 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.522960 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.528635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.528606 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 15:47:20.528635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.528638 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kgb8h\"" Jul 14 15:47:20.529802 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.529630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 15:47:20.535211 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.535186 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 15:47:20.535819 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.535792 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 15:47:20.536577 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.536546 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sq9l9"] Jul 14 15:47:20.536681 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.536665 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.539338 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.539310 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-km4cd\"" Jul 14 15:47:20.539672 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.539650 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 15:47:20.540000 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.539981 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 15:47:20.554364 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.554343 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77cf6fc8b9-89rdg"] Jul 14 15:47:20.554470 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.554382 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d7652"] Jul 14 15:47:20.554470 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.554412 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sq9l9"] Jul 14 15:47:20.554550 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.554504 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.556989 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.556968 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9twdb\"" Jul 14 15:47:20.557117 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.556989 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 15:47:20.557913 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.557893 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 15:47:20.557994 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.557922 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 15:47:20.666432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8da8bdc-83e2-4baf-a20d-fe118ad3541e-cert\") pod \"ingress-canary-sq9l9\" (UID: \"d8da8bdc-83e2-4baf-a20d-fe118ad3541e\") " pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.666582 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666463 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-tmp-dir\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.666582 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666582 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptrlg\" (UniqueName: \"kubernetes.io/projected/d8da8bdc-83e2-4baf-a20d-fe118ad3541e-kube-api-access-ptrlg\") pod \"ingress-canary-sq9l9\" (UID: \"d8da8bdc-83e2-4baf-a20d-fe118ad3541e\") " pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.666690 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-config-volume\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.666690 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666652 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cfb2\" (UniqueName: \"kubernetes.io/projected/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-kube-api-access-6cfb2\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.666690 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666672 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-certificates\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.666800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666725 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-trusted-ca\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666751 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-image-registry-private-configuration\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666781 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-installation-pull-secrets\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666806 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-bound-sa-token\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/918930b7-b625-4dcb-b760-b07fbd8c93ad-ca-trust-extracted\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.666967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.666891 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smfd\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-kube-api-access-7smfd\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.767824 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767739 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptrlg\" (UniqueName: \"kubernetes.io/projected/d8da8bdc-83e2-4baf-a20d-fe118ad3541e-kube-api-access-ptrlg\") pod \"ingress-canary-sq9l9\" (UID: \"d8da8bdc-83e2-4baf-a20d-fe118ad3541e\") " pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.767824 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-config-volume\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.767824 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767813 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cfb2\" (UniqueName: \"kubernetes.io/projected/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-kube-api-access-6cfb2\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767841 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-certificates\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767868 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-trusted-ca\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767926 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-image-registry-private-configuration\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-installation-pull-secrets\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.767977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-bound-sa-token\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/918930b7-b625-4dcb-b760-b07fbd8c93ad-ca-trust-extracted\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smfd\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-kube-api-access-7smfd\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8da8bdc-83e2-4baf-a20d-fe118ad3541e-cert\") pod \"ingress-canary-sq9l9\" (UID: \"d8da8bdc-83e2-4baf-a20d-fe118ad3541e\") " pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:20.768099 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768124 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-tmp-dir\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.768155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768156 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768756 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:20.768220 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls podName:ebcfae7d-263f-4a16-860a-1bc56f7ee88c nodeName:}" failed. No retries permitted until 2026-07-14 15:47:21.268201741 +0000 UTC m=+36.654289109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls") pod "dns-default-d7652" (UID: "ebcfae7d-263f-4a16-860a-1bc56f7ee88c") : secret "dns-default-metrics-tls" not found Jul 14 15:47:20.768756 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768518 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-config-volume\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.768756 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768607 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-certificates\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768756 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768703 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/918930b7-b625-4dcb-b760-b07fbd8c93ad-ca-trust-extracted\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.768955 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:20.768774 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 15:47:20.768955 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:20.768783 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77cf6fc8b9-89rdg: secret "image-registry-tls" not found Jul 14 15:47:20.768955 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768786 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-tmp-dir\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.768955 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:20.768817 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls podName:918930b7-b625-4dcb-b760-b07fbd8c93ad nodeName:}" failed. No retries permitted until 2026-07-14 15:47:21.268803612 +0000 UTC m=+36.654890977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls") pod "image-registry-77cf6fc8b9-89rdg" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad") : secret "image-registry-tls" not found Jul 14 15:47:20.768955 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.768921 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-trusted-ca\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.773098 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.773075 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-installation-pull-secrets\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.773229 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.773084 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-image-registry-private-configuration\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.773229 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.773117 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8da8bdc-83e2-4baf-a20d-fe118ad3541e-cert\") pod \"ingress-canary-sq9l9\" (UID: \"d8da8bdc-83e2-4baf-a20d-fe118ad3541e\") " pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.780007 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.779948 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smfd\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-kube-api-access-7smfd\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.784227 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.784204 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-bound-sa-token\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:20.784754 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.784736 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cfb2\" (UniqueName: \"kubernetes.io/projected/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-kube-api-access-6cfb2\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:20.785403 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.785384 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptrlg\" (UniqueName: \"kubernetes.io/projected/d8da8bdc-83e2-4baf-a20d-fe118ad3541e-kube-api-access-ptrlg\") pod \"ingress-canary-sq9l9\" (UID: \"d8da8bdc-83e2-4baf-a20d-fe118ad3541e\") " pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.863727 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.863695 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sq9l9" Jul 14 15:47:20.899032 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.898972 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-srdcg"] Jul 14 15:47:20.914945 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.914912 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-srdcg"] Jul 14 15:47:20.915143 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.915016 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:20.917889 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.917707 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5hh22\"" Jul 14 15:47:20.918031 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.917926 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 14 15:47:20.918031 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:20.918016 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 14 15:47:21.070589 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.070508 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:21.070743 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.070592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d680f3e0-33f9-430a-ad55-b8839d32415e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:21.092731 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.092696 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:21.092731 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.092730 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:21.092950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.092703 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:21.096374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.095381 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 15:47:21.096374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.095442 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt8h7\"" Jul 14 15:47:21.096374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.095451 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 15:47:21.096374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.095713 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfvj8\"" Jul 14 15:47:21.096374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.095907 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 15:47:21.096374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.096128 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 15:47:21.171565 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.171527 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:21.171779 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.171616 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d680f3e0-33f9-430a-ad55-b8839d32415e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:21.171779 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.171709 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 15:47:21.171888 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.171805 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert podName:d680f3e0-33f9-430a-ad55-b8839d32415e nodeName:}" failed. No retries permitted until 2026-07-14 15:47:21.671781148 +0000 UTC m=+37.057868515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-srdcg" (UID: "d680f3e0-33f9-430a-ad55-b8839d32415e") : secret "networking-console-plugin-cert" not found Jul 14 15:47:21.172426 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.172401 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d680f3e0-33f9-430a-ad55-b8839d32415e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:21.272835 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.272794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:21.273035 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.272909 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:21.273035 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.272968 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 15:47:21.273035 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.273018 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 15:47:21.273035 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.273032 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77cf6fc8b9-89rdg: secret "image-registry-tls" not found Jul 14 15:47:21.273265 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.273075 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls podName:ebcfae7d-263f-4a16-860a-1bc56f7ee88c nodeName:}" failed. No retries permitted until 2026-07-14 15:47:22.273030995 +0000 UTC m=+37.659118381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls") pod "dns-default-d7652" (UID: "ebcfae7d-263f-4a16-860a-1bc56f7ee88c") : secret "dns-default-metrics-tls" not found Jul 14 15:47:21.273265 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.273099 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls podName:918930b7-b625-4dcb-b760-b07fbd8c93ad nodeName:}" failed. No retries permitted until 2026-07-14 15:47:22.273087215 +0000 UTC m=+37.659174581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls") pod "image-registry-77cf6fc8b9-89rdg" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad") : secret "image-registry-tls" not found Jul 14 15:47:21.375074 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.375030 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4v2r5"] Jul 14 15:47:21.406493 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.406464 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.411864 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.411842 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 15:47:21.412727 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.412416 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 15:47:21.412727 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.412487 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 15:47:21.412727 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.412489 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lf679\"" Jul 14 15:47:21.412727 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.412514 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 15:47:21.421128 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.421097 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4v2r5"] Jul 14 15:47:21.527510 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.527483 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sq9l9"] Jul 14 15:47:21.533838 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:21.533793 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8da8bdc_83e2_4baf_a20d_fe118ad3541e.slice/crio-82c8878f8dbe321dc6a3435455042695407bc6341900a3718dd88846e06e79f2 WatchSource:0}: Error finding container 82c8878f8dbe321dc6a3435455042695407bc6341900a3718dd88846e06e79f2: Status 404 returned error can't find the container with id 82c8878f8dbe321dc6a3435455042695407bc6341900a3718dd88846e06e79f2 Jul 14 15:47:21.574942 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.574914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-data-volume\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.575235 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.574944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg7bf\" (UniqueName: \"kubernetes.io/projected/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-api-access-cg7bf\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.575235 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.575010 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.575235 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.575054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-crio-socket\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.575235 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.575100 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676270 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676245 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:21.676346 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-data-volume\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676346 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7bf\" (UniqueName: \"kubernetes.io/projected/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-api-access-cg7bf\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676424 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.676400 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 15:47:21.676486 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.676475 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert podName:d680f3e0-33f9-430a-ad55-b8839d32415e nodeName:}" failed. No retries permitted until 2026-07-14 15:47:22.676457005 +0000 UTC m=+38.062544373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-srdcg" (UID: "d680f3e0-33f9-430a-ad55-b8839d32415e") : secret "networking-console-plugin-cert" not found Jul 14 15:47:21.676540 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676513 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676579 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676554 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-crio-socket\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676628 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676674 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.676658 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 15:47:21.676723 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.676697 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 15:47:21.676723 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676709 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-data-volume\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.676805 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.676724 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:22.176706866 +0000 UTC m=+37.562794243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : configmap "kube-rbac-proxy" not found Jul 14 15:47:21.676805 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:21.676748 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:22.176734969 +0000 UTC m=+37.562822338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : secret "insights-runtime-extractor-tls" not found Jul 14 15:47:21.676872 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.676815 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-crio-socket\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:21.688566 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:21.688542 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg7bf\" (UniqueName: \"kubernetes.io/projected/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-api-access-cg7bf\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:22.180631 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.180586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:22.180818 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.180728 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 15:47:22.180818 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.180743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:22.180818 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.180802 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:23.18078068 +0000 UTC m=+38.566868048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : secret "insights-runtime-extractor-tls" not found Jul 14 15:47:22.181010 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.180832 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 15:47:22.181010 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.180882 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:23.180866496 +0000 UTC m=+38.566953865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : configmap "kube-rbac-proxy" not found Jul 14 15:47:22.185867 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.185835 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx"] Jul 14 15:47:22.187947 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.187924 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" Jul 14 15:47:22.191203 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.191178 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 14 15:47:22.191553 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.191532 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 14 15:47:22.191667 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.191568 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-sjzth\"" Jul 14 15:47:22.203862 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.203834 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx"] Jul 14 15:47:22.281779 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.281743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:22.281938 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.281819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcvjm\" (UniqueName: \"kubernetes.io/projected/4a02d65d-9638-4c31-adb2-f839a29bbbc2-kube-api-access-rcvjm\") pod \"migrator-7f59b54f6c-gkrsx\" (UID: \"4a02d65d-9638-4c31-adb2-f839a29bbbc2\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" Jul 14 15:47:22.281938 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.281919 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 15:47:22.282080 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.282010 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls podName:ebcfae7d-263f-4a16-860a-1bc56f7ee88c nodeName:}" failed. No retries permitted until 2026-07-14 15:47:24.281978068 +0000 UTC m=+39.668065448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls") pod "dns-default-d7652" (UID: "ebcfae7d-263f-4a16-860a-1bc56f7ee88c") : secret "dns-default-metrics-tls" not found Jul 14 15:47:22.282080 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.282055 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:22.282207 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.282191 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 15:47:22.282254 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.282210 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77cf6fc8b9-89rdg: secret "image-registry-tls" not found Jul 14 15:47:22.282302 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.282255 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls podName:918930b7-b625-4dcb-b760-b07fbd8c93ad nodeName:}" failed. No retries permitted until 2026-07-14 15:47:24.282241869 +0000 UTC m=+39.668329250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls") pod "image-registry-77cf6fc8b9-89rdg" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad") : secret "image-registry-tls" not found Jul 14 15:47:22.341180 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.341142 2573 generic.go:356] "Generic (PLEG): container finished" podID="c581dd53-95b4-46a5-bd65-87722abac294" containerID="a378cdd58279dfa1378e5d5cc85130318edc178e2699fcbdf5aac9228167eea3" exitCode=0 Jul 14 15:47:22.341353 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.341232 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerDied","Data":"a378cdd58279dfa1378e5d5cc85130318edc178e2699fcbdf5aac9228167eea3"} Jul 14 15:47:22.342434 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.342412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sq9l9" event={"ID":"d8da8bdc-83e2-4baf-a20d-fe118ad3541e","Type":"ContainerStarted","Data":"82c8878f8dbe321dc6a3435455042695407bc6341900a3718dd88846e06e79f2"} Jul 14 15:47:22.383368 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.383332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcvjm\" (UniqueName: \"kubernetes.io/projected/4a02d65d-9638-4c31-adb2-f839a29bbbc2-kube-api-access-rcvjm\") pod \"migrator-7f59b54f6c-gkrsx\" (UID: \"4a02d65d-9638-4c31-adb2-f839a29bbbc2\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" Jul 14 15:47:22.393370 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.393342 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcvjm\" (UniqueName: \"kubernetes.io/projected/4a02d65d-9638-4c31-adb2-f839a29bbbc2-kube-api-access-rcvjm\") pod \"migrator-7f59b54f6c-gkrsx\" (UID: \"4a02d65d-9638-4c31-adb2-f839a29bbbc2\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" Jul 14 15:47:22.499883 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.499855 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" Jul 14 15:47:22.660397 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.660364 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx"] Jul 14 15:47:22.664697 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:22.664669 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a02d65d_9638_4c31_adb2_f839a29bbbc2.slice/crio-0a1956fbf928a01eed712dde92f51a807fe5bda89b04819c012cfac9d44b4d34 WatchSource:0}: Error finding container 0a1956fbf928a01eed712dde92f51a807fe5bda89b04819c012cfac9d44b4d34: Status 404 returned error can't find the container with id 0a1956fbf928a01eed712dde92f51a807fe5bda89b04819c012cfac9d44b4d34 Jul 14 15:47:22.686096 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.686016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:22.686217 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.686149 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 15:47:22.686217 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:22.686212 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert podName:d680f3e0-33f9-430a-ad55-b8839d32415e nodeName:}" failed. No retries permitted until 2026-07-14 15:47:24.686192025 +0000 UTC m=+40.072279415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-srdcg" (UID: "d680f3e0-33f9-430a-ad55-b8839d32415e") : secret "networking-console-plugin-cert" not found Jul 14 15:47:22.748975 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:22.748947 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g6vvg_bd82b629-05a6-4aa2-9a72-1c421ae37e0d/dns-node-resolver/0.log" Jul 14 15:47:23.191083 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.190799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:23.191270 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.191132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:23.191315 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:23.190940 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 15:47:23.191373 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:23.191355 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:25.191336794 +0000 UTC m=+40.577424180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : configmap "kube-rbac-proxy" not found Jul 14 15:47:23.191373 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:23.191289 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 15:47:23.191516 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:23.191447 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:25.191425366 +0000 UTC m=+40.577512746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : secret "insights-runtime-extractor-tls" not found Jul 14 15:47:23.349320 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.349283 2573 generic.go:356] "Generic (PLEG): container finished" podID="c581dd53-95b4-46a5-bd65-87722abac294" containerID="1f6d80c1f9348dc4a2ac59839de7c91c97a73cfa3d9059275c2178a0c1ba692e" exitCode=0 Jul 14 15:47:23.349496 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.349365 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerDied","Data":"1f6d80c1f9348dc4a2ac59839de7c91c97a73cfa3d9059275c2178a0c1ba692e"} Jul 14 15:47:23.350723 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.350693 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" event={"ID":"4a02d65d-9638-4c31-adb2-f839a29bbbc2","Type":"ContainerStarted","Data":"0a1956fbf928a01eed712dde92f51a807fe5bda89b04819c012cfac9d44b4d34"} Jul 14 15:47:23.748851 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.748816 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hpv8z_ae7ada8b-e8d8-489d-bf68-b3b820771c1a/node-ca/0.log" Jul 14 15:47:23.888079 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.888024 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-t7nqv"] Jul 14 15:47:23.890188 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.890162 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:23.892758 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.892738 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 15:47:23.892862 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.892781 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 15:47:23.892862 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.892740 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 15:47:23.892994 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.892978 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bkvgq\"" Jul 14 15:47:23.893137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.893025 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 15:47:23.898930 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.898897 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-t7nqv"] Jul 14 15:47:23.998343 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.998304 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-signing-cabundle\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:23.998511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.998380 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwb4k\" (UniqueName: \"kubernetes.io/projected/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-kube-api-access-lwb4k\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:23.998511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:23.998448 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-signing-key\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.099275 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.099189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-signing-cabundle\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.099275 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.099271 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwb4k\" (UniqueName: \"kubernetes.io/projected/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-kube-api-access-lwb4k\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.099507 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.099391 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-signing-key\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.100035 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.100005 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-signing-cabundle\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.103409 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.103390 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-signing-key\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.112722 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.112687 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwb4k\" (UniqueName: \"kubernetes.io/projected/c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3-kube-api-access-lwb4k\") pod \"service-ca-79f68b9f4f-t7nqv\" (UID: \"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3\") " pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.203266 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.203245 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" Jul 14 15:47:24.302461 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.302382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:24.303349 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.302782 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:24.303349 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.302934 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 15:47:24.303349 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.302938 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 15:47:24.303349 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.302949 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77cf6fc8b9-89rdg: secret "image-registry-tls" not found Jul 14 15:47:24.303349 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.302997 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls podName:ebcfae7d-263f-4a16-860a-1bc56f7ee88c nodeName:}" failed. No retries permitted until 2026-07-14 15:47:28.302976118 +0000 UTC m=+43.689063503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls") pod "dns-default-d7652" (UID: "ebcfae7d-263f-4a16-860a-1bc56f7ee88c") : secret "dns-default-metrics-tls" not found Jul 14 15:47:24.303349 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.303014 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls podName:918930b7-b625-4dcb-b760-b07fbd8c93ad nodeName:}" failed. No retries permitted until 2026-07-14 15:47:28.303005521 +0000 UTC m=+43.689092889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls") pod "image-registry-77cf6fc8b9-89rdg" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad") : secret "image-registry-tls" not found Jul 14 15:47:24.352569 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.352342 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-t7nqv"] Jul 14 15:47:24.356335 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:24.356302 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63aa0dc_95d7_4ea7_8ca5_396cd27fc7b3.slice/crio-262e3c142228c2cbbbeeb4e1ebbdb47c77f926c1dc99d36511ef15c8181868aa WatchSource:0}: Error finding container 262e3c142228c2cbbbeeb4e1ebbdb47c77f926c1dc99d36511ef15c8181868aa: Status 404 returned error can't find the container with id 262e3c142228c2cbbbeeb4e1ebbdb47c77f926c1dc99d36511ef15c8181868aa Jul 14 15:47:24.358115 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.358081 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hklg2" event={"ID":"c581dd53-95b4-46a5-bd65-87722abac294","Type":"ContainerStarted","Data":"fa7c59812f256b48710166cec463b23a5f0f47295c651a36f1d28bd1b120c721"} Jul 14 15:47:24.359686 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.359656 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sq9l9" event={"ID":"d8da8bdc-83e2-4baf-a20d-fe118ad3541e","Type":"ContainerStarted","Data":"c0dc7553977fa1c5794beb6aeb1ed610609c2d8e25def34fc3f7c9038fffeb36"} Jul 14 15:47:24.361602 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.361551 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" event={"ID":"4a02d65d-9638-4c31-adb2-f839a29bbbc2","Type":"ContainerStarted","Data":"593a7701be9c1ae03eac525d8301d0baf3d4e3581610d3db1562633ed476babb"} Jul 14 15:47:24.388188 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.388141 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hklg2" podStartSLOduration=7.622385307 podStartE2EDuration="39.388125936s" totalImagesPullingTime="31.765740629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:24.387508388 +0000 UTC m=+39.773595775" watchObservedRunningTime="2026-07-14 15:47:24.388125936 +0000 UTC m=+39.774213322" Jul 14 15:47:24.403944 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.403894 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sq9l9" podStartSLOduration=1.741738716 podStartE2EDuration="4.403878485s" totalImagesPullingTime="2.662139769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:24.403231836 +0000 UTC m=+39.789319239" watchObservedRunningTime="2026-07-14 15:47:24.403878485 +0000 UTC m=+39.789965920" Jul 14 15:47:24.706411 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.706376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:24.706560 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.706535 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 15:47:24.706611 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:24.706599 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert podName:d680f3e0-33f9-430a-ad55-b8839d32415e nodeName:}" failed. No retries permitted until 2026-07-14 15:47:28.706583313 +0000 UTC m=+44.092670678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-srdcg" (UID: "d680f3e0-33f9-430a-ad55-b8839d32415e") : secret "networking-console-plugin-cert" not found Jul 14 15:47:24.748801 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:24.748771 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sq9l9_d8da8bdc-83e2-4baf-a20d-fe118ad3541e/serve-healthcheck-canary/0.log" Jul 14 15:47:25.210244 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:25.210205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:25.211015 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:25.210355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:25.211015 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:25.210368 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 15:47:25.211015 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:25.210430 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:29.21041328 +0000 UTC m=+44.596500666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : secret "insights-runtime-extractor-tls" not found Jul 14 15:47:25.211015 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:25.210438 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 15:47:25.211015 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:25.210471 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:29.210461235 +0000 UTC m=+44.596548608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : configmap "kube-rbac-proxy" not found Jul 14 15:47:25.366368 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:25.366335 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" event={"ID":"4a02d65d-9638-4c31-adb2-f839a29bbbc2","Type":"ContainerStarted","Data":"664f4b5ff77cf30096a806e026aa22b793a03549a3bb0e64c4f5f5029d7b2b3f"} Jul 14 15:47:25.367649 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:25.367618 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" event={"ID":"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3","Type":"ContainerStarted","Data":"262e3c142228c2cbbbeeb4e1ebbdb47c77f926c1dc99d36511ef15c8181868aa"} Jul 14 15:47:25.388187 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:25.388143 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gkrsx" podStartSLOduration=1.8598263720000001 podStartE2EDuration="3.388123168s" totalImagesPullingTime="1.528296796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:25.387076944 +0000 UTC m=+40.773164350" watchObservedRunningTime="2026-07-14 15:47:25.388123168 +0000 UTC m=+40.774210562" Jul 14 15:47:27.376379 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:27.376335 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" event={"ID":"c63aa0dc-95d7-4ea7-8ca5-396cd27fc7b3","Type":"ContainerStarted","Data":"554c8efd2f5bb36469fda53bb1c41f2c0568814a726dbd19a224f226d25c06a8"} Jul 14 15:47:27.400954 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:27.400902 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-t7nqv" podStartSLOduration=2.300478771 podStartE2EDuration="4.400886553s" totalImagesPullingTime="2.100407782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:27.400789257 +0000 UTC m=+42.786876673" watchObservedRunningTime="2026-07-14 15:47:27.400886553 +0000 UTC m=+42.786973988" Jul 14 15:47:28.335334 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:28.335298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:28.335527 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:28.335356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:28.335527 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.335443 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 15:47:28.335527 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.335461 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77cf6fc8b9-89rdg: secret "image-registry-tls" not found Jul 14 15:47:28.335527 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.335501 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 15:47:28.335527 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.335515 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls podName:918930b7-b625-4dcb-b760-b07fbd8c93ad nodeName:}" failed. No retries permitted until 2026-07-14 15:47:36.335501512 +0000 UTC m=+51.721588909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls") pod "image-registry-77cf6fc8b9-89rdg" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad") : secret "image-registry-tls" not found Jul 14 15:47:28.335692 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.335544 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls podName:ebcfae7d-263f-4a16-860a-1bc56f7ee88c nodeName:}" failed. No retries permitted until 2026-07-14 15:47:36.33553354 +0000 UTC m=+51.721620905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls") pod "dns-default-d7652" (UID: "ebcfae7d-263f-4a16-860a-1bc56f7ee88c") : secret "dns-default-metrics-tls" not found Jul 14 15:47:28.739429 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:28.739387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:28.739877 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.739535 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 15:47:28.739877 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:28.739600 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert podName:d680f3e0-33f9-430a-ad55-b8839d32415e nodeName:}" failed. No retries permitted until 2026-07-14 15:47:36.739585012 +0000 UTC m=+52.125672376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-srdcg" (UID: "d680f3e0-33f9-430a-ad55-b8839d32415e") : secret "networking-console-plugin-cert" not found Jul 14 15:47:29.244155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:29.244120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:29.244342 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:29.244166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:29.244342 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:29.244260 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 15:47:29.244342 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:47:29.244311 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls podName:6bdfe042-b0b5-4ebc-8c6d-8414d737327a nodeName:}" failed. No retries permitted until 2026-07-14 15:47:37.244298335 +0000 UTC m=+52.630385700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4v2r5" (UID: "6bdfe042-b0b5-4ebc-8c6d-8414d737327a") : secret "insights-runtime-extractor-tls" not found Jul 14 15:47:29.244656 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:29.244635 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:36.402287 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.402247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:36.402740 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.402332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:36.406131 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.406101 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"image-registry-77cf6fc8b9-89rdg\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:36.420734 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.420710 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ebcfae7d-263f-4a16-860a-1bc56f7ee88c-metrics-tls\") pod \"dns-default-d7652\" (UID: \"ebcfae7d-263f-4a16-860a-1bc56f7ee88c\") " pod="openshift-dns/dns-default-d7652" Jul 14 15:47:36.433660 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.433631 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:36.447515 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.447495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d7652" Jul 14 15:47:36.607222 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.607190 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77cf6fc8b9-89rdg"] Jul 14 15:47:36.610154 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:36.610129 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918930b7_b625_4dcb_b760_b07fbd8c93ad.slice/crio-36e2be8f01d252f6d9ba27295d00c0372145e23881e88ad3dfb5c2a28886f491 WatchSource:0}: Error finding container 36e2be8f01d252f6d9ba27295d00c0372145e23881e88ad3dfb5c2a28886f491: Status 404 returned error can't find the container with id 36e2be8f01d252f6d9ba27295d00c0372145e23881e88ad3dfb5c2a28886f491 Jul 14 15:47:36.626151 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.626123 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d7652"] Jul 14 15:47:36.630465 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:36.630441 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebcfae7d_263f_4a16_860a_1bc56f7ee88c.slice/crio-6aa4e17b922d5a0e455028d9b3f3ff81f91a6dbb7ae2057711de4ea10551227f WatchSource:0}: Error finding container 6aa4e17b922d5a0e455028d9b3f3ff81f91a6dbb7ae2057711de4ea10551227f: Status 404 returned error can't find the container with id 6aa4e17b922d5a0e455028d9b3f3ff81f91a6dbb7ae2057711de4ea10551227f Jul 14 15:47:36.805188 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.805151 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:36.807585 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.807562 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d680f3e0-33f9-430a-ad55-b8839d32415e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-srdcg\" (UID: \"d680f3e0-33f9-430a-ad55-b8839d32415e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:36.826477 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.826454 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" Jul 14 15:47:36.981032 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:36.979434 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-srdcg"] Jul 14 15:47:36.987738 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:36.987693 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd680f3e0_33f9_430a_ad55_b8839d32415e.slice/crio-d5f6453045b976b08e8aa12945dc2316d273188e4c18882fe0c453d597f81df0 WatchSource:0}: Error finding container d5f6453045b976b08e8aa12945dc2316d273188e4c18882fe0c453d597f81df0: Status 404 returned error can't find the container with id d5f6453045b976b08e8aa12945dc2316d273188e4c18882fe0c453d597f81df0 Jul 14 15:47:37.309434 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.309325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:37.311683 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.311664 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6bdfe042-b0b5-4ebc-8c6d-8414d737327a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4v2r5\" (UID: \"6bdfe042-b0b5-4ebc-8c6d-8414d737327a\") " pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:37.315724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.315709 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4v2r5" Jul 14 15:47:37.404223 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.404184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" event={"ID":"d680f3e0-33f9-430a-ad55-b8839d32415e","Type":"ContainerStarted","Data":"d5f6453045b976b08e8aa12945dc2316d273188e4c18882fe0c453d597f81df0"} Jul 14 15:47:37.405808 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.405780 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" event={"ID":"918930b7-b625-4dcb-b760-b07fbd8c93ad","Type":"ContainerStarted","Data":"76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08"} Jul 14 15:47:37.405953 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.405820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" event={"ID":"918930b7-b625-4dcb-b760-b07fbd8c93ad","Type":"ContainerStarted","Data":"36e2be8f01d252f6d9ba27295d00c0372145e23881e88ad3dfb5c2a28886f491"} Jul 14 15:47:37.405953 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.405866 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:47:37.407092 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.407031 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7652" event={"ID":"ebcfae7d-263f-4a16-860a-1bc56f7ee88c","Type":"ContainerStarted","Data":"6aa4e17b922d5a0e455028d9b3f3ff81f91a6dbb7ae2057711de4ea10551227f"} Jul 14 15:47:37.434383 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.434329 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" podStartSLOduration=48.434311938 podStartE2EDuration="48.434311938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:37.432868909 +0000 UTC m=+52.818956297" watchObservedRunningTime="2026-07-14 15:47:37.434311938 +0000 UTC m=+52.820399324" Jul 14 15:47:37.472318 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:37.472286 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4v2r5"] Jul 14 15:47:37.475254 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:37.475226 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdfe042_b0b5_4ebc_8c6d_8414d737327a.slice/crio-8f84a0ea4831ed22aaf7e3c637ec4db0ff7ceddb3a6553a605ee4c65fc3d2c46 WatchSource:0}: Error finding container 8f84a0ea4831ed22aaf7e3c637ec4db0ff7ceddb3a6553a605ee4c65fc3d2c46: Status 404 returned error can't find the container with id 8f84a0ea4831ed22aaf7e3c637ec4db0ff7ceddb3a6553a605ee4c65fc3d2c46 Jul 14 15:47:38.411650 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:38.411615 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v2r5" event={"ID":"6bdfe042-b0b5-4ebc-8c6d-8414d737327a","Type":"ContainerStarted","Data":"8f84a0ea4831ed22aaf7e3c637ec4db0ff7ceddb3a6553a605ee4c65fc3d2c46"} Jul 14 15:47:39.418717 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:39.418669 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" event={"ID":"d680f3e0-33f9-430a-ad55-b8839d32415e","Type":"ContainerStarted","Data":"db8d7443b59e71a0ade217c86592246ec92277cddbf9675ee5acf395566de055"} Jul 14 15:47:43.434736 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:43.434701 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7652" event={"ID":"ebcfae7d-263f-4a16-860a-1bc56f7ee88c","Type":"ContainerStarted","Data":"c1d57ace185ad0fe31f69f1d7a5b3d1c66ec1def5732034648113c55e93a2d02"} Jul 14 15:47:43.435202 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:43.434760 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7652" event={"ID":"ebcfae7d-263f-4a16-860a-1bc56f7ee88c","Type":"ContainerStarted","Data":"cf01146d83e0400342b68bf00763ab853de975aecced6a611e553c0f3bc23c6c"} Jul 14 15:47:43.435202 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:43.434818 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d7652" Jul 14 15:47:43.436228 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:43.436205 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v2r5" event={"ID":"6bdfe042-b0b5-4ebc-8c6d-8414d737327a","Type":"ContainerStarted","Data":"652623a379ffbb899e518165a2f20d273e7dff552f0faab5ac6c0b1f18099a1b"} Jul 14 15:47:43.458915 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:43.458867 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-srdcg" podStartSLOduration=21.503611509 podStartE2EDuration="23.458852078s" totalImagesPullingTime="1.955240569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:39.435740306 +0000 UTC m=+54.821827691" watchObservedRunningTime="2026-07-14 15:47:43.458852078 +0000 UTC m=+58.844939466" Jul 14 15:47:43.459212 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:43.459173 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-d7652" podStartSLOduration=17.15962309 podStartE2EDuration="23.459165046s" totalImagesPullingTime="6.299541956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:43.458999656 +0000 UTC m=+58.845087034" watchObservedRunningTime="2026-07-14 15:47:43.459165046 +0000 UTC m=+58.845252435" Jul 14 15:47:45.446223 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:45.446185 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v2r5" event={"ID":"6bdfe042-b0b5-4ebc-8c6d-8414d737327a","Type":"ContainerStarted","Data":"5c18c2e99135bf448c7cc13f2c566e4e25bbeb942e1438fd8d9b41ef9ae71e31"} Jul 14 15:47:46.340750 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:46.340722 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bffbw" Jul 14 15:47:47.136894 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.136863 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng"] Jul 14 15:47:47.177269 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.177233 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw"] Jul 14 15:47:47.177528 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.177379 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.181411 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.181385 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 15:47:47.181411 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.181433 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 15:47:47.181638 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.181388 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 15:47:47.182407 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.182224 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 15:47:47.189890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.189870 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng"] Jul 14 15:47:47.190020 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.189964 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.195473 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.195116 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 14 15:47:47.196110 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.195661 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw"] Jul 14 15:47:47.196110 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.195837 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 14 15:47:47.196319 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.196248 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 14 15:47:47.196611 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.196591 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 14 15:47:47.239714 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.239685 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77cf6fc8b9-89rdg"] Jul 14 15:47:47.286062 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.285994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-klusterlet-config\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.286246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286069 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.286246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-ca\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.286246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56xnt\" (UniqueName: \"kubernetes.io/projected/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-kube-api-access-56xnt\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.286246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-hub\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.286246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286202 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-tmp\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.286246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c397c815-b6dd-483f-902f-1868e7b4adec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.286554 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286259 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.286554 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.286284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64c64\" (UniqueName: \"kubernetes.io/projected/c397c815-b6dd-483f-902f-1868e7b4adec-kube-api-access-64c64\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.330488 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.330457 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bf6456885-v88g8"] Jul 14 15:47:47.366421 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.366389 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bf6456885-v88g8"] Jul 14 15:47:47.366599 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.366518 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.387303 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387219 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffda826d-2c0c-49ad-9087-1dc19b74c902-ca-trust-extracted\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.387303 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387271 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-klusterlet-config\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.387303 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.387541 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387360 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-ca\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.387541 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387392 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56xnt\" (UniqueName: \"kubernetes.io/projected/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-kube-api-access-56xnt\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.387758 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387732 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffda826d-2c0c-49ad-9087-1dc19b74c902-trusted-ca\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.387892 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-hub\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.387892 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387822 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4jww\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-kube-api-access-h4jww\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.387892 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387851 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-tmp\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.387892 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387876 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c397c815-b6dd-483f-902f-1868e7b4adec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.388137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.388137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64c64\" (UniqueName: \"kubernetes.io/projected/c397c815-b6dd-483f-902f-1868e7b4adec-kube-api-access-64c64\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.388137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387960 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-registry-tls\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.388137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.387986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffda826d-2c0c-49ad-9087-1dc19b74c902-registry-certificates\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.388137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.388015 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffda826d-2c0c-49ad-9087-1dc19b74c902-image-registry-private-configuration\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.388137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.388074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-bound-sa-token\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.388437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.388273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffda826d-2c0c-49ad-9087-1dc19b74c902-installation-pull-secrets\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.389285 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.389250 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-tmp\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.389405 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.389318 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c397c815-b6dd-483f-902f-1868e7b4adec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.390489 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.390457 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-ca\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.390734 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.390711 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.391368 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.391344 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-hub\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.391572 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.391551 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c397c815-b6dd-483f-902f-1868e7b4adec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.391895 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.391878 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-klusterlet-config\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.402601 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.402565 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56xnt\" (UniqueName: \"kubernetes.io/projected/2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6-kube-api-access-56xnt\") pod \"klusterlet-addon-workmgr-778df7cb47-zzbng\" (UID: \"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.405953 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.405922 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64c64\" (UniqueName: \"kubernetes.io/projected/c397c815-b6dd-483f-902f-1868e7b4adec-kube-api-access-64c64\") pod \"cluster-proxy-proxy-agent-5cdccbf468-vxcjw\" (UID: \"c397c815-b6dd-483f-902f-1868e7b4adec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.489596 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:47.489781 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489686 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffda826d-2c0c-49ad-9087-1dc19b74c902-installation-pull-secrets\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489781 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffda826d-2c0c-49ad-9087-1dc19b74c902-ca-trust-extracted\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489887 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489814 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffda826d-2c0c-49ad-9087-1dc19b74c902-trusted-ca\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489887 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489841 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4jww\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-kube-api-access-h4jww\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489887 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489875 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-registry-tls\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489986 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489898 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffda826d-2c0c-49ad-9087-1dc19b74c902-registry-certificates\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489986 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffda826d-2c0c-49ad-9087-1dc19b74c902-image-registry-private-configuration\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.489986 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.489943 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-bound-sa-token\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.491967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.490955 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffda826d-2c0c-49ad-9087-1dc19b74c902-ca-trust-extracted\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.491967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.491744 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffda826d-2c0c-49ad-9087-1dc19b74c902-registry-certificates\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.491967 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.491884 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffda826d-2c0c-49ad-9087-1dc19b74c902-trusted-ca\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.493354 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.493330 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-registry-tls\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.493682 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.493635 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffda826d-2c0c-49ad-9087-1dc19b74c902-installation-pull-secrets\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.493919 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.493896 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffda826d-2c0c-49ad-9087-1dc19b74c902-image-registry-private-configuration\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.511951 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.511921 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" Jul 14 15:47:47.512340 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.512315 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4jww\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-kube-api-access-h4jww\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.523923 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.523898 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffda826d-2c0c-49ad-9087-1dc19b74c902-bound-sa-token\") pod \"image-registry-5bf6456885-v88g8\" (UID: \"ffda826d-2c0c-49ad-9087-1dc19b74c902\") " pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.678003 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.677927 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:47.987677 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:47.987646 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw"] Jul 14 15:47:47.990867 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:47.990838 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc397c815_b6dd_483f_902f_1868e7b4adec.slice/crio-73bf54264611de799938139c4853c3aed1bb50e6c9b4534597b2a304bbf29c32 WatchSource:0}: Error finding container 73bf54264611de799938139c4853c3aed1bb50e6c9b4534597b2a304bbf29c32: Status 404 returned error can't find the container with id 73bf54264611de799938139c4853c3aed1bb50e6c9b4534597b2a304bbf29c32 Jul 14 15:47:48.189204 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.189119 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng"] Jul 14 15:47:48.192155 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.192131 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bf6456885-v88g8"] Jul 14 15:47:48.192707 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:48.192683 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7e6b4f_2ac1_40c7_9ca6_c47a43607af6.slice/crio-2e1caf7d9547ea41d0d50ac8cb368fda7b6e4937b2a9f01ad01903d78f5d43dd WatchSource:0}: Error finding container 2e1caf7d9547ea41d0d50ac8cb368fda7b6e4937b2a9f01ad01903d78f5d43dd: Status 404 returned error can't find the container with id 2e1caf7d9547ea41d0d50ac8cb368fda7b6e4937b2a9f01ad01903d78f5d43dd Jul 14 15:47:48.195466 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:48.195445 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffda826d_2c0c_49ad_9087_1dc19b74c902.slice/crio-fecc8642a56439e94ae0bc5ad77e18b8f78849d20f95c5d79af2edc52dc88224 WatchSource:0}: Error finding container fecc8642a56439e94ae0bc5ad77e18b8f78849d20f95c5d79af2edc52dc88224: Status 404 returned error can't find the container with id fecc8642a56439e94ae0bc5ad77e18b8f78849d20f95c5d79af2edc52dc88224 Jul 14 15:47:48.459649 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.459565 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" event={"ID":"ffda826d-2c0c-49ad-9087-1dc19b74c902","Type":"ContainerStarted","Data":"ce171e12321c92c0b566e735d0bffda682bf271fec0b934a0b7b17e26ae0dc66"} Jul 14 15:47:48.459649 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.459627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" event={"ID":"ffda826d-2c0c-49ad-9087-1dc19b74c902","Type":"ContainerStarted","Data":"fecc8642a56439e94ae0bc5ad77e18b8f78849d20f95c5d79af2edc52dc88224"} Jul 14 15:47:48.460270 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.460255 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:47:48.461391 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.461359 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" event={"ID":"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6","Type":"ContainerStarted","Data":"2e1caf7d9547ea41d0d50ac8cb368fda7b6e4937b2a9f01ad01903d78f5d43dd"} Jul 14 15:47:48.464238 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.464186 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4v2r5" event={"ID":"6bdfe042-b0b5-4ebc-8c6d-8414d737327a","Type":"ContainerStarted","Data":"a54cf4d16ab1dbc6d1a63ede5797bb4bb195e36e93180f6be7a81f66a49414ee"} Jul 14 15:47:48.466818 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.466786 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" event={"ID":"c397c815-b6dd-483f-902f-1868e7b4adec","Type":"ContainerStarted","Data":"73bf54264611de799938139c4853c3aed1bb50e6c9b4534597b2a304bbf29c32"} Jul 14 15:47:48.494405 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.490697 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" podStartSLOduration=1.490674216 podStartE2EDuration="1.490674216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:48.490344004 +0000 UTC m=+63.876431390" watchObservedRunningTime="2026-07-14 15:47:48.490674216 +0000 UTC m=+63.876761605" Jul 14 15:47:48.527822 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:48.527759 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4v2r5" podStartSLOduration=17.292082112 podStartE2EDuration="27.527734059s" totalImagesPullingTime="10.235651947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:47:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:48.526625268 +0000 UTC m=+63.912712658" watchObservedRunningTime="2026-07-14 15:47:48.527734059 +0000 UTC m=+63.913821438" Jul 14 15:47:49.811457 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.811417 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:49.814586 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.814557 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 15:47:49.827472 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.827409 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8-original-pull-secret\") pod \"global-pull-secret-syncer-bg8f6\" (UID: \"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8\") " pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:49.912763 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.912720 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:49.917740 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.917713 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 15:47:49.918790 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.918763 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg8f6" Jul 14 15:47:49.925945 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:49.925919 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d96c9827-c29c-4426-927a-e9710682b720-metrics-certs\") pod \"network-metrics-daemon-l5rh8\" (UID: \"d96c9827-c29c-4426-927a-e9710682b720\") " pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:50.013806 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.013717 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:50.017122 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.016993 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 15:47:50.027332 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.027182 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 15:47:50.038200 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.038176 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbnx9\" (UniqueName: \"kubernetes.io/projected/14573d3c-2ea9-4b67-8494-1981ea25e12b-kube-api-access-dbnx9\") pod \"network-check-target-8l64k\" (UID: \"14573d3c-2ea9-4b67-8494-1981ea25e12b\") " pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:50.104244 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.104207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bg8f6"] Jul 14 15:47:50.108126 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:50.108093 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ba13aec_0ae1_4d1c_873e_e2ffba8b0da8.slice/crio-9b2643e1fbc50a2ef06b109e757341630764c8f668d29212d33a04be9e1e919e WatchSource:0}: Error finding container 9b2643e1fbc50a2ef06b109e757341630764c8f668d29212d33a04be9e1e919e: Status 404 returned error can't find the container with id 9b2643e1fbc50a2ef06b109e757341630764c8f668d29212d33a04be9e1e919e Jul 14 15:47:50.208908 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.208834 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt8h7\"" Jul 14 15:47:50.215987 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.215795 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfvj8\"" Jul 14 15:47:50.216249 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.216230 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:50.224082 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.223995 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5rh8" Jul 14 15:47:50.475420 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:50.475328 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bg8f6" event={"ID":"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8","Type":"ContainerStarted","Data":"9b2643e1fbc50a2ef06b109e757341630764c8f668d29212d33a04be9e1e919e"} Jul 14 15:47:51.664692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:51.664663 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l5rh8"] Jul 14 15:47:51.669180 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:47:51.669129 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96c9827_c29c_4426_927a_e9710682b720.slice/crio-4c03859e8c78a110e63d3cfa39b53465b44e58de570328cfc0db78a368c0f877 WatchSource:0}: Error finding container 4c03859e8c78a110e63d3cfa39b53465b44e58de570328cfc0db78a368c0f877: Status 404 returned error can't find the container with id 4c03859e8c78a110e63d3cfa39b53465b44e58de570328cfc0db78a368c0f877 Jul 14 15:47:51.681763 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:51.681678 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8l64k"] Jul 14 15:47:52.485132 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:52.485091 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8l64k" event={"ID":"14573d3c-2ea9-4b67-8494-1981ea25e12b","Type":"ContainerStarted","Data":"8a413d262b918c8dd74aee9bb7404e4bc0a75f0ac3008931dd679c5fc24c6391"} Jul 14 15:47:52.487794 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:52.487759 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" event={"ID":"c397c815-b6dd-483f-902f-1868e7b4adec","Type":"ContainerStarted","Data":"7d762226c541f3e316c7e99ce65a3f3b8a0d153adc6320a07d39fcfc5d90871c"} Jul 14 15:47:52.487948 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:52.487812 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" event={"ID":"c397c815-b6dd-483f-902f-1868e7b4adec","Type":"ContainerStarted","Data":"9abf4fed62f8d8fdb975243ad5f37504f161efc3e3b7fd543be3197d43790e26"} Jul 14 15:47:52.487948 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:52.487834 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" event={"ID":"c397c815-b6dd-483f-902f-1868e7b4adec","Type":"ContainerStarted","Data":"1640d1f73c6266ba726c3c5ebbb76ea26d5a2d0341bb20a5d4af7e3b0c463715"} Jul 14 15:47:52.489307 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:52.489279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l5rh8" event={"ID":"d96c9827-c29c-4426-927a-e9710682b720","Type":"ContainerStarted","Data":"4c03859e8c78a110e63d3cfa39b53465b44e58de570328cfc0db78a368c0f877"} Jul 14 15:47:52.528869 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:52.528797 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cdccbf468-vxcjw" podStartSLOduration=1.986667264 podStartE2EDuration="5.528773727s" totalImagesPullingTime="3.542106463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:52.526970183 +0000 UTC m=+67.913057577" watchObservedRunningTime="2026-07-14 15:47:52.528773727 +0000 UTC m=+67.914861096" Jul 14 15:47:53.442174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:53.442139 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d7652" Jul 14 15:47:54.505259 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:54.505218 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" event={"ID":"2d7e6b4f-2ac1-40c7-9ca6-c47a43607af6","Type":"ContainerStarted","Data":"e423d7263631bafd025d3e9b4658c2e1fa65ee7d4691171ee954f7e7e893e244"} Jul 14 15:47:54.506245 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:54.506213 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:54.508263 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:54.508239 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" Jul 14 15:47:54.527236 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:54.527177 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778df7cb47-zzbng" podStartSLOduration=2.260657266 podStartE2EDuration="7.527154869s" totalImagesPullingTime="5.266497603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:54.526498085 +0000 UTC m=+69.912585472" watchObservedRunningTime="2026-07-14 15:47:54.527154869 +0000 UTC m=+69.913242258" Jul 14 15:47:57.245420 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:57.245382 2573 patch_prober.go:28] interesting pod/image-registry-77cf6fc8b9-89rdg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 15:47:57.245758 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:57.245441 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" podUID="918930b7-b625-4dcb-b760-b07fbd8c93ad" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 15:47:58.521304 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.521264 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bg8f6" event={"ID":"1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8","Type":"ContainerStarted","Data":"181dda4f438dd983b4ab138c1bd75abbf9432972a76d5e55bb87dfd48b5549d3"} Jul 14 15:47:58.522724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.522699 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8l64k" event={"ID":"14573d3c-2ea9-4b67-8494-1981ea25e12b","Type":"ContainerStarted","Data":"7b1544e10e5f265e1381987c8c5820d5d4f971f780f17fdb5263aaae2cadfa11"} Jul 14 15:47:58.522801 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.522784 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:47:58.524386 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.524361 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l5rh8" event={"ID":"d96c9827-c29c-4426-927a-e9710682b720","Type":"ContainerStarted","Data":"e2e1ab05322bb94657f50755d9bc155b6497db60adc6c9c6d237fdf96436aac6"} Jul 14 15:47:58.524491 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.524410 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l5rh8" event={"ID":"d96c9827-c29c-4426-927a-e9710682b720","Type":"ContainerStarted","Data":"28d1f2092e7c7ad829775d0d402a8e69663e1cd4aae4e68a4d2e7b4861d17743"} Jul 14 15:47:58.541698 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.541653 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bg8f6" podStartSLOduration=66.217906277 podStartE2EDuration="1m13.541634008s" totalImagesPullingTime="7.323727731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:58.540856222 +0000 UTC m=+73.926943610" watchObservedRunningTime="2026-07-14 15:47:58.541634008 +0000 UTC m=+73.927721398" Jul 14 15:47:58.562736 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.562689 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l5rh8" podStartSLOduration=67.802807695 podStartE2EDuration="1m13.562672529s" totalImagesPullingTime="5.759864834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:58.561529769 +0000 UTC m=+73.947617157" watchObservedRunningTime="2026-07-14 15:47:58.562672529 +0000 UTC m=+73.948759921" Jul 14 15:47:58.580631 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:47:58.580577 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8l64k" podStartSLOduration=67.824984327 podStartE2EDuration="1m13.580564595s" totalImagesPullingTime="5.755580268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:58.5797374 +0000 UTC m=+73.965824807" watchObservedRunningTime="2026-07-14 15:47:58.580564595 +0000 UTC m=+73.966652037" Jul 14 15:48:03.111095 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.111058 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nk82g"] Jul 14 15:48:03.113637 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.113613 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.121136 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.121106 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 15:48:03.121825 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.121794 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 15:48:03.121944 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.121880 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vsgwq\"" Jul 14 15:48:03.122182 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.122164 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 15:48:03.122182 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.122177 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 15:48:03.122358 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.122229 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 15:48:03.122472 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.122443 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 15:48:03.222337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222305 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-textfile\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222345 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-tls\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b03483f6-8589-4b95-947f-8379c4af2b8b-metrics-client-ca\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222408 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-wtmp\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzrgr\" (UniqueName: \"kubernetes.io/projected/b03483f6-8589-4b95-947f-8379c4af2b8b-kube-api-access-lzrgr\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222473 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-sys\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222542 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-root\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.222744 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.222582 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323285 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323221 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-textfile\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323285 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323271 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-tls\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b03483f6-8589-4b95-947f-8379c4af2b8b-metrics-client-ca\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-wtmp\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323369 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzrgr\" (UniqueName: \"kubernetes.io/projected/b03483f6-8589-4b95-947f-8379c4af2b8b-kube-api-access-lzrgr\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323394 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-sys\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323422 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-root\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323847 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323565 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-wtmp\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323847 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323649 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-sys\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.323847 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.323707 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b03483f6-8589-4b95-947f-8379c4af2b8b-root\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.324023 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.324000 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.324231 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.324208 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b03483f6-8589-4b95-947f-8379c4af2b8b-metrics-client-ca\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.324461 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.324439 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-textfile\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.326330 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.326304 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.326757 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.326735 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b03483f6-8589-4b95-947f-8379c4af2b8b-node-exporter-tls\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.332253 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.332228 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzrgr\" (UniqueName: \"kubernetes.io/projected/b03483f6-8589-4b95-947f-8379c4af2b8b-kube-api-access-lzrgr\") pod \"node-exporter-nk82g\" (UID: \"b03483f6-8589-4b95-947f-8379c4af2b8b\") " pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.424210 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.424179 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nk82g" Jul 14 15:48:03.434795 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:48:03.434768 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03483f6_8589_4b95_947f_8379c4af2b8b.slice/crio-4e3d4dc8a623afdd8c937c0935bdc377f47bb823d402f5ab42928b5c23d8bded WatchSource:0}: Error finding container 4e3d4dc8a623afdd8c937c0935bdc377f47bb823d402f5ab42928b5c23d8bded: Status 404 returned error can't find the container with id 4e3d4dc8a623afdd8c937c0935bdc377f47bb823d402f5ab42928b5c23d8bded Jul 14 15:48:03.542441 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:03.542401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nk82g" event={"ID":"b03483f6-8589-4b95-947f-8379c4af2b8b","Type":"ContainerStarted","Data":"4e3d4dc8a623afdd8c937c0935bdc377f47bb823d402f5ab42928b5c23d8bded"} Jul 14 15:48:05.550963 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:05.550924 2573 generic.go:356] "Generic (PLEG): container finished" podID="b03483f6-8589-4b95-947f-8379c4af2b8b" containerID="a2590bb830753e3b25b477f942b0321e04497d31dd470b5ff47a588ec296833b" exitCode=0 Jul 14 15:48:05.551345 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:05.551022 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nk82g" event={"ID":"b03483f6-8589-4b95-947f-8379c4af2b8b","Type":"ContainerDied","Data":"a2590bb830753e3b25b477f942b0321e04497d31dd470b5ff47a588ec296833b"} Jul 14 15:48:06.558581 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:06.558550 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nk82g" event={"ID":"b03483f6-8589-4b95-947f-8379c4af2b8b","Type":"ContainerStarted","Data":"251368cb3842182754f242947bb8b2c309828fb051ea3078421d4836d6e26b20"} Jul 14 15:48:06.559064 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:06.558603 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nk82g" event={"ID":"b03483f6-8589-4b95-947f-8379c4af2b8b","Type":"ContainerStarted","Data":"9a80412445ef655ba9f654d1d534f3d1fcb0a4840985d0925c19bd2d672831ab"} Jul 14 15:48:06.584774 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:06.584717 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nk82g" podStartSLOduration=2.064665849 podStartE2EDuration="3.584700101s" totalImagesPullingTime="1.446536534s" totalInitContainerRuntime="73.497718ms" isStatefulPod=true podCreationTimestamp="2026-07-14 15:48:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:06.58295876 +0000 UTC m=+81.969046184" watchObservedRunningTime="2026-07-14 15:48:06.584700101 +0000 UTC m=+81.970787487" Jul 14 15:48:07.244490 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:07.244456 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:48:10.480179 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:10.480152 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bf6456885-v88g8" Jul 14 15:48:12.261840 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.261780 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" podUID="918930b7-b625-4dcb-b760-b07fbd8c93ad" containerName="registry" containerID="cri-o://76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08" gracePeriod=30 Jul 14 15:48:12.495535 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.495511 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:48:12.586813 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.586724 2573 generic.go:356] "Generic (PLEG): container finished" podID="918930b7-b625-4dcb-b760-b07fbd8c93ad" containerID="76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08" exitCode=0 Jul 14 15:48:12.586813 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.586785 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" Jul 14 15:48:12.587016 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.586786 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" event={"ID":"918930b7-b625-4dcb-b760-b07fbd8c93ad","Type":"ContainerDied","Data":"76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08"} Jul 14 15:48:12.587016 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.586920 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77cf6fc8b9-89rdg" event={"ID":"918930b7-b625-4dcb-b760-b07fbd8c93ad","Type":"ContainerDied","Data":"36e2be8f01d252f6d9ba27295d00c0372145e23881e88ad3dfb5c2a28886f491"} Jul 14 15:48:12.587016 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.586941 2573 scope.go:117] "RemoveContainer" containerID="76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08" Jul 14 15:48:12.595817 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.595801 2573 scope.go:117] "RemoveContainer" containerID="76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08" Jul 14 15:48:12.596098 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:48:12.596077 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08\": container with ID starting with 76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08 not found: ID does not exist" containerID="76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08" Jul 14 15:48:12.596153 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.596109 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08"} err="failed to get container status \"76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08\": rpc error: code = NotFound desc = could not find container \"76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08\": container with ID starting with 76289fb383e90335736de504a3c15b8d2f58e7345471434900e10ccbb40c4e08 not found: ID does not exist" Jul 14 15:48:12.600344 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600328 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600402 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600374 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-installation-pull-secrets\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600439 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600400 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-image-registry-private-configuration\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600439 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600431 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-bound-sa-token\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600514 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600455 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/918930b7-b625-4dcb-b760-b07fbd8c93ad-ca-trust-extracted\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600514 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600507 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-trusted-ca\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600622 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600542 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7smfd\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-kube-api-access-7smfd\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.600622 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.600569 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-certificates\") pod \"918930b7-b625-4dcb-b760-b07fbd8c93ad\" (UID: \"918930b7-b625-4dcb-b760-b07fbd8c93ad\") " Jul 14 15:48:12.601342 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.601314 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 15:48:12.601500 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.601426 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 15:48:12.603501 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.603476 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:48:12.603607 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.603529 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:48:12.603607 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.603579 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:48:12.603838 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.603817 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:48:12.603880 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.603856 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-kube-api-access-7smfd" (OuterVolumeSpecName: "kube-api-access-7smfd") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "kube-api-access-7smfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:48:12.612327 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.612296 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/918930b7-b625-4dcb-b760-b07fbd8c93ad-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "918930b7-b625-4dcb-b760-b07fbd8c93ad" (UID: "918930b7-b625-4dcb-b760-b07fbd8c93ad"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 15:48:12.701983 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.701946 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-trusted-ca\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.701983 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.701979 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7smfd\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-kube-api-access-7smfd\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.701983 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.701989 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-certificates\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.702217 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.701997 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-registry-tls\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.702217 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.702006 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-installation-pull-secrets\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.702217 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.702016 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/918930b7-b625-4dcb-b760-b07fbd8c93ad-image-registry-private-configuration\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.702217 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.702025 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/918930b7-b625-4dcb-b760-b07fbd8c93ad-bound-sa-token\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.702217 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.702033 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/918930b7-b625-4dcb-b760-b07fbd8c93ad-ca-trust-extracted\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:48:12.907339 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.907310 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77cf6fc8b9-89rdg"] Jul 14 15:48:12.913472 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:12.913448 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77cf6fc8b9-89rdg"] Jul 14 15:48:13.097643 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:13.097601 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="918930b7-b625-4dcb-b760-b07fbd8c93ad" path="/var/lib/kubelet/pods/918930b7-b625-4dcb-b760-b07fbd8c93ad/volumes" Jul 14 15:48:29.530694 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:48:29.530663 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8l64k" Jul 14 15:51:44.998746 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:51:44.998718 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:51:45.000251 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:51:45.000227 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:51:45.006191 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:51:45.006155 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 15:52:57.078252 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.078172 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-t4nnv"] Jul 14 15:52:57.078635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.078468 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="918930b7-b625-4dcb-b760-b07fbd8c93ad" containerName="registry" Jul 14 15:52:57.078635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.078482 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="918930b7-b625-4dcb-b760-b07fbd8c93ad" containerName="registry" Jul 14 15:52:57.078635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.078528 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="918930b7-b625-4dcb-b760-b07fbd8c93ad" containerName="registry" Jul 14 15:52:57.080208 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.080192 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.082439 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.082412 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 15:52:57.082611 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.082450 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sz8xw\"" Jul 14 15:52:57.083213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.083200 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 15:52:57.092062 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.092020 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-t4nnv"] Jul 14 15:52:57.170940 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.170905 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0707e381-5597-4aac-abd7-71f05e66f709-bound-sa-token\") pod \"cert-manager-55b4d954d5-t4nnv\" (UID: \"0707e381-5597-4aac-abd7-71f05e66f709\") " pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.171138 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.170954 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm4kf\" (UniqueName: \"kubernetes.io/projected/0707e381-5597-4aac-abd7-71f05e66f709-kube-api-access-jm4kf\") pod \"cert-manager-55b4d954d5-t4nnv\" (UID: \"0707e381-5597-4aac-abd7-71f05e66f709\") " pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.272195 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.272151 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0707e381-5597-4aac-abd7-71f05e66f709-bound-sa-token\") pod \"cert-manager-55b4d954d5-t4nnv\" (UID: \"0707e381-5597-4aac-abd7-71f05e66f709\") " pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.272195 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.272202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm4kf\" (UniqueName: \"kubernetes.io/projected/0707e381-5597-4aac-abd7-71f05e66f709-kube-api-access-jm4kf\") pod \"cert-manager-55b4d954d5-t4nnv\" (UID: \"0707e381-5597-4aac-abd7-71f05e66f709\") " pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.280832 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.280795 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0707e381-5597-4aac-abd7-71f05e66f709-bound-sa-token\") pod \"cert-manager-55b4d954d5-t4nnv\" (UID: \"0707e381-5597-4aac-abd7-71f05e66f709\") " pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.280955 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.280877 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm4kf\" (UniqueName: \"kubernetes.io/projected/0707e381-5597-4aac-abd7-71f05e66f709-kube-api-access-jm4kf\") pod \"cert-manager-55b4d954d5-t4nnv\" (UID: \"0707e381-5597-4aac-abd7-71f05e66f709\") " pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.388413 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.388383 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-t4nnv" Jul 14 15:52:57.523180 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.523151 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-t4nnv"] Jul 14 15:52:57.526063 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:52:57.526016 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0707e381_5597_4aac_abd7_71f05e66f709.slice/crio-d69c9d91526f8f37632e30866557a2b13ec278cbaf3b2d94df5725ad55f0d197 WatchSource:0}: Error finding container d69c9d91526f8f37632e30866557a2b13ec278cbaf3b2d94df5725ad55f0d197: Status 404 returned error can't find the container with id d69c9d91526f8f37632e30866557a2b13ec278cbaf3b2d94df5725ad55f0d197 Jul 14 15:52:57.528295 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.528270 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 15:52:57.547555 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:52:57.547531 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-t4nnv" event={"ID":"0707e381-5597-4aac-abd7-71f05e66f709","Type":"ContainerStarted","Data":"d69c9d91526f8f37632e30866557a2b13ec278cbaf3b2d94df5725ad55f0d197"} Jul 14 15:53:01.570449 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:01.570409 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-t4nnv" event={"ID":"0707e381-5597-4aac-abd7-71f05e66f709","Type":"ContainerStarted","Data":"a5e6f8179a2249c16ba5a2c07c47c71029ccfa0e2ef48959f83c12c0c02d5952"} Jul 14 15:53:01.590695 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:01.590650 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-t4nnv" podStartSLOduration=1.576465222 podStartE2EDuration="4.590636455s" totalImagesPullingTime="3.014171233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:01.589093608 +0000 UTC m=+376.975181017" watchObservedRunningTime="2026-07-14 15:53:01.590636455 +0000 UTC m=+376.976723842" Jul 14 15:53:14.355589 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.355554 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz"] Jul 14 15:53:14.361320 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.361298 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.363739 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.363717 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 15:53:14.363857 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.363830 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 15:53:14.363857 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.363835 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 15:53:14.363980 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.363927 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-klszc\"" Jul 14 15:53:14.363980 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.363948 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 15:53:14.372300 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.372277 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz"] Jul 14 15:53:14.506539 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.506499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebd8ad92-e085-4264-9d16-687d7140b148-manifests\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.506539 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.506542 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebd8ad92-e085-4264-9d16-687d7140b148-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.506800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.506606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebd8ad92-e085-4264-9d16-687d7140b148-webhook-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.506800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.506655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebd8ad92-e085-4264-9d16-687d7140b148-tmp\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.506800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.506685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcd8n\" (UniqueName: \"kubernetes.io/projected/ebd8ad92-e085-4264-9d16-687d7140b148-kube-api-access-tcd8n\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.607501 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.607410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebd8ad92-e085-4264-9d16-687d7140b148-webhook-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.607501 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.607462 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebd8ad92-e085-4264-9d16-687d7140b148-tmp\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.607501 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.607496 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcd8n\" (UniqueName: \"kubernetes.io/projected/ebd8ad92-e085-4264-9d16-687d7140b148-kube-api-access-tcd8n\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.607746 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.607535 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebd8ad92-e085-4264-9d16-687d7140b148-manifests\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.607746 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.607570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebd8ad92-e085-4264-9d16-687d7140b148-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.608098 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.608067 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ebd8ad92-e085-4264-9d16-687d7140b148-manifests\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.610534 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.610360 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebd8ad92-e085-4264-9d16-687d7140b148-tmp\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.610534 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.610491 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ebd8ad92-e085-4264-9d16-687d7140b148-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.610938 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.610909 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebd8ad92-e085-4264-9d16-687d7140b148-webhook-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.618737 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.618702 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcd8n\" (UniqueName: \"kubernetes.io/projected/ebd8ad92-e085-4264-9d16-687d7140b148-kube-api-access-tcd8n\") pod \"opendatahub-operator-controller-manager-7486689d9d-j4xqz\" (UID: \"ebd8ad92-e085-4264-9d16-687d7140b148\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.672184 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.672156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:14.811812 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:14.811780 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz"] Jul 14 15:53:14.815925 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:14.815896 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd8ad92_e085_4264_9d16_687d7140b148.slice/crio-055af547ffd257279e006f54cf9c3580abc998bf026dec163ca8413f6961800d WatchSource:0}: Error finding container 055af547ffd257279e006f54cf9c3580abc998bf026dec163ca8413f6961800d: Status 404 returned error can't find the container with id 055af547ffd257279e006f54cf9c3580abc998bf026dec163ca8413f6961800d Jul 14 15:53:15.620833 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:15.620794 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" event={"ID":"ebd8ad92-e085-4264-9d16-687d7140b148","Type":"ContainerStarted","Data":"055af547ffd257279e006f54cf9c3580abc998bf026dec163ca8413f6961800d"} Jul 14 15:53:18.634885 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:18.634850 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" event={"ID":"ebd8ad92-e085-4264-9d16-687d7140b148","Type":"ContainerStarted","Data":"c34fa664dc0c2636d594de49a10486ec2f0d94229e3e1dd4b609db99ebdded3c"} Jul 14 15:53:19.641030 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:19.640992 2573 generic.go:356] "Generic (PLEG): container finished" podID="ebd8ad92-e085-4264-9d16-687d7140b148" containerID="c34fa664dc0c2636d594de49a10486ec2f0d94229e3e1dd4b609db99ebdded3c" exitCode=0 Jul 14 15:53:19.641530 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:19.641073 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" event={"ID":"ebd8ad92-e085-4264-9d16-687d7140b148","Type":"ContainerDied","Data":"c34fa664dc0c2636d594de49a10486ec2f0d94229e3e1dd4b609db99ebdded3c"} Jul 14 15:53:20.647839 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:20.647795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" event={"ID":"ebd8ad92-e085-4264-9d16-687d7140b148","Type":"ContainerStarted","Data":"562b8c8a3f7f96a5f8ee817c26d19de67536af5b80c02999d73c3121c93e6906"} Jul 14 15:53:20.648362 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:20.647969 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:20.668903 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:20.668835 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" podStartSLOduration=1.976975347 podStartE2EDuration="6.668813347s" totalImagesPullingTime="3.612158113s" totalInitContainerRuntime="1.079679887s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:20.667304654 +0000 UTC m=+396.053392060" watchObservedRunningTime="2026-07-14 15:53:20.668813347 +0000 UTC m=+396.054900737" Jul 14 15:53:31.653564 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:31.653531 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-j4xqz" Jul 14 15:53:34.796066 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.796007 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5"] Jul 14 15:53:34.804111 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.804085 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.807073 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.807026 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 15:53:34.807717 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.807696 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 15:53:34.807838 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.807703 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 15:53:34.807838 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.807825 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 15:53:34.807952 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.807865 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8mx9f\"" Jul 14 15:53:34.814301 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.814276 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5"] Jul 14 15:53:34.864905 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.864871 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bdec269-762f-48a9-aa6b-0da3f21cf27f-tls-certs\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.865115 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.864921 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bdec269-762f-48a9-aa6b-0da3f21cf27f-tmp\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.865115 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.864983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc8gr\" (UniqueName: \"kubernetes.io/projected/0bdec269-762f-48a9-aa6b-0da3f21cf27f-kube-api-access-dc8gr\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.965411 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.965367 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n88b6"] Jul 14 15:53:34.965645 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.965622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bdec269-762f-48a9-aa6b-0da3f21cf27f-tmp\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.965724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.965687 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc8gr\" (UniqueName: \"kubernetes.io/projected/0bdec269-762f-48a9-aa6b-0da3f21cf27f-kube-api-access-dc8gr\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.965778 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.965740 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bdec269-762f-48a9-aa6b-0da3f21cf27f-tls-certs\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.968299 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.968270 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bdec269-762f-48a9-aa6b-0da3f21cf27f-tmp\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.968421 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.968398 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bdec269-762f-48a9-aa6b-0da3f21cf27f-tls-certs\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:34.968701 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.968682 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:34.971393 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.971371 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-w9s67\"" Jul 14 15:53:34.971517 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.971473 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 14 15:53:34.986127 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.986091 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n88b6"] Jul 14 15:53:34.988596 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:34.988537 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc8gr\" (UniqueName: \"kubernetes.io/projected/0bdec269-762f-48a9-aa6b-0da3f21cf27f-kube-api-access-dc8gr\") pod \"kube-auth-proxy-85568bb86c-t4vn5\" (UID: \"0bdec269-762f-48a9-aa6b-0da3f21cf27f\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:35.066340 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.066243 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn6h4\" (UniqueName: \"kubernetes.io/projected/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-kube-api-access-dn6h4\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.066340 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.066288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-cert\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.116057 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.116010 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" Jul 14 15:53:35.167331 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.167305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-cert\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.167523 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:53:35.167460 2573 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 14 15:53:35.167592 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.167525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn6h4\" (UniqueName: \"kubernetes.io/projected/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-kube-api-access-dn6h4\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.167746 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:53:35.167715 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-cert podName:8caf40ed-7cb1-451b-9f94-f4081b24f9ce nodeName:}" failed. No retries permitted until 2026-07-14 15:53:35.6676903 +0000 UTC m=+411.053777668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-cert") pod "odh-model-controller-7998c67dc4-n88b6" (UID: "8caf40ed-7cb1-451b-9f94-f4081b24f9ce") : secret "odh-model-controller-webhook-cert" not found Jul 14 15:53:35.182858 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.182831 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn6h4\" (UniqueName: \"kubernetes.io/projected/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-kube-api-access-dn6h4\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.286752 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.286714 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5"] Jul 14 15:53:35.290930 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:35.290881 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdec269_762f_48a9_aa6b_0da3f21cf27f.slice/crio-f1018f07ecc8fccf453552cc5630dc31d7d889e36a92bd3e6e3e94cf50bf7952 WatchSource:0}: Error finding container f1018f07ecc8fccf453552cc5630dc31d7d889e36a92bd3e6e3e94cf50bf7952: Status 404 returned error can't find the container with id f1018f07ecc8fccf453552cc5630dc31d7d889e36a92bd3e6e3e94cf50bf7952 Jul 14 15:53:35.671063 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.671010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-cert\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.673812 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.673783 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8caf40ed-7cb1-451b-9f94-f4081b24f9ce-cert\") pod \"odh-model-controller-7998c67dc4-n88b6\" (UID: \"8caf40ed-7cb1-451b-9f94-f4081b24f9ce\") " pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.706173 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.706136 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" event={"ID":"0bdec269-762f-48a9-aa6b-0da3f21cf27f","Type":"ContainerStarted","Data":"f1018f07ecc8fccf453552cc5630dc31d7d889e36a92bd3e6e3e94cf50bf7952"} Jul 14 15:53:35.901857 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.901805 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:35.911074 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.911022 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g52jn"] Jul 14 15:53:35.914448 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.914427 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:35.917924 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.917897 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nwnpl\"" Jul 14 15:53:35.918859 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.918830 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 15:53:35.928652 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.928626 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g52jn"] Jul 14 15:53:35.970580 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.970436 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx"] Jul 14 15:53:35.972406 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.972269 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b-tls-certs\") pod \"model-serving-api-54d89bd5fc-g52jn\" (UID: \"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b\") " pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:35.972406 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.972311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5gph\" (UniqueName: \"kubernetes.io/projected/f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b-kube-api-access-r5gph\") pod \"model-serving-api-54d89bd5fc-g52jn\" (UID: \"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b\") " pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:35.975165 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.973939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:35.977215 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.976535 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 15:53:35.977215 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.976821 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 15:53:35.977215 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.976998 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bz77g\"" Jul 14 15:53:35.977913 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.977599 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 15:53:35.977913 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.977773 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 15:53:35.989690 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:35.989662 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx"] Jul 14 15:53:36.069245 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.069001 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n88b6"] Jul 14 15:53:36.072242 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:36.072190 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8caf40ed_7cb1_451b_9f94_f4081b24f9ce.slice/crio-c864f49314a149114aee8d1247287a978381f4e95e960e71ce37a97af2f1ffea WatchSource:0}: Error finding container c864f49314a149114aee8d1247287a978381f4e95e960e71ce37a97af2f1ffea: Status 404 returned error can't find the container with id c864f49314a149114aee8d1247287a978381f4e95e960e71ce37a97af2f1ffea Jul 14 15:53:36.073123 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5gph\" (UniqueName: \"kubernetes.io/projected/f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b-kube-api-access-r5gph\") pod \"model-serving-api-54d89bd5fc-g52jn\" (UID: \"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b\") " pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:36.073260 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073136 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073260 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsq4z\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-kube-api-access-hsq4z\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073260 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073247 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073401 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073270 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073401 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073291 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-token\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073401 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-local-certs\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073525 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-cacerts\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.073525 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.073475 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b-tls-certs\") pod \"model-serving-api-54d89bd5fc-g52jn\" (UID: \"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b\") " pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:36.076362 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.076342 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b-tls-certs\") pod \"model-serving-api-54d89bd5fc-g52jn\" (UID: \"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b\") " pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:36.084842 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.084811 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5gph\" (UniqueName: \"kubernetes.io/projected/f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b-kube-api-access-r5gph\") pod \"model-serving-api-54d89bd5fc-g52jn\" (UID: \"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b\") " pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:36.174856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.174756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.175020 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.174959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsq4z\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-kube-api-access-hsq4z\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.175137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.175026 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.175137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.175082 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.175137 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.175120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-token\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.175296 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.175178 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-local-certs\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.175630 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.175602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-cacerts\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.176070 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.176016 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.178133 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.178104 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.178348 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.178314 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-local-certs\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.178444 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.178404 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-cacerts\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.178610 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.178595 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.188014 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.187987 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-token\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.189947 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.189919 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsq4z\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-kube-api-access-hsq4z\") pod \"istiod-openshift-gateway-7574459656-qrmrx\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.226288 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.226253 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:36.298367 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.298332 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:36.383828 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.383785 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g52jn"] Jul 14 15:53:36.389838 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:36.389804 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b1b776_a609_4b8a_ae2f_cfa12aa8b41b.slice/crio-4be380cb9cf24de042b29fd357f8a20754b75fd4f787ccf4923e5e1d246550a9 WatchSource:0}: Error finding container 4be380cb9cf24de042b29fd357f8a20754b75fd4f787ccf4923e5e1d246550a9: Status 404 returned error can't find the container with id 4be380cb9cf24de042b29fd357f8a20754b75fd4f787ccf4923e5e1d246550a9 Jul 14 15:53:36.462103 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.461982 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx"] Jul 14 15:53:36.465751 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:36.465715 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0f0a46_5e36_4bbd_8f47_da8e6e949ceb.slice/crio-d392cdf873f73e63dab61904120ebf13f361ad9d1fc5a1500068d77af7dc714f WatchSource:0}: Error finding container d392cdf873f73e63dab61904120ebf13f361ad9d1fc5a1500068d77af7dc714f: Status 404 returned error can't find the container with id d392cdf873f73e63dab61904120ebf13f361ad9d1fc5a1500068d77af7dc714f Jul 14 15:53:36.712681 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.712594 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" event={"ID":"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b","Type":"ContainerStarted","Data":"4be380cb9cf24de042b29fd357f8a20754b75fd4f787ccf4923e5e1d246550a9"} Jul 14 15:53:36.713999 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.713969 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" event={"ID":"8caf40ed-7cb1-451b-9f94-f4081b24f9ce","Type":"ContainerStarted","Data":"c864f49314a149114aee8d1247287a978381f4e95e960e71ce37a97af2f1ffea"} Jul 14 15:53:36.715338 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:36.715308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" event={"ID":"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb","Type":"ContainerStarted","Data":"d392cdf873f73e63dab61904120ebf13f361ad9d1fc5a1500068d77af7dc714f"} Jul 14 15:53:41.427843 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.427809 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:53:41.428180 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.427882 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:53:41.747247 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.747135 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" event={"ID":"f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b","Type":"ContainerStarted","Data":"3862a0248cf8f2edc562ec79ca53134d59bc5426463f20ad2777ee59bb38aac8"} Jul 14 15:53:41.747408 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.747268 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:41.748905 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.748878 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" event={"ID":"0bdec269-762f-48a9-aa6b-0da3f21cf27f","Type":"ContainerStarted","Data":"97238b1c2c5266aa38b111e98877b6eb84beb79406c83a9d50bb3b6d0e286a89"} Jul 14 15:53:41.750595 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.750572 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" event={"ID":"8caf40ed-7cb1-451b-9f94-f4081b24f9ce","Type":"ContainerStarted","Data":"7538752d26ce11a3abf9cf7a2009e305d5d01bc12f8aead2c35ef49f1e7e78e1"} Jul 14 15:53:41.750734 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.750643 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:41.752288 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.752263 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" event={"ID":"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb","Type":"ContainerStarted","Data":"b21ddeeaa243dc1e2ef4f74fe99a86daaa20e8119b85386d0be236f49846d466"} Jul 14 15:53:41.752422 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.752358 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:41.765505 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.765457 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" podStartSLOduration=1.692393708 podStartE2EDuration="6.765440378s" totalImagesPullingTime="5.07304667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:41.764500608 +0000 UTC m=+417.150588023" watchObservedRunningTime="2026-07-14 15:53:41.765440378 +0000 UTC m=+417.151527768" Jul 14 15:53:41.783421 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.783374 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85568bb86c-t4vn5" podStartSLOduration=1.6523742399999999 podStartE2EDuration="7.783359702s" totalImagesPullingTime="6.130985462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:41.78178802 +0000 UTC m=+417.167875419" watchObservedRunningTime="2026-07-14 15:53:41.783359702 +0000 UTC m=+417.169447097" Jul 14 15:53:41.808361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.808295 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" podStartSLOduration=1.849051243 podStartE2EDuration="6.808272059s" totalImagesPullingTime="4.959220816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:41.807778293 +0000 UTC m=+417.193865681" watchObservedRunningTime="2026-07-14 15:53:41.808272059 +0000 UTC m=+417.194359447" Jul 14 15:53:41.827216 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:41.827165 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" podStartSLOduration=2.444189072 podStartE2EDuration="7.827144971s" totalImagesPullingTime="5.382955899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:41.826225281 +0000 UTC m=+417.212312670" watchObservedRunningTime="2026-07-14 15:53:41.827144971 +0000 UTC m=+417.213232357" Jul 14 15:53:42.758466 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:42.758440 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:47.426944 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.426912 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg"] Jul 14 15:53:47.431557 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.431533 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.438589 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.438245 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-lvqrs\"" Jul 14 15:53:47.438589 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.438369 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 14 15:53:47.446246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.446171 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg"] Jul 14 15:53:47.458617 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.458586 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx"] Jul 14 15:53:47.462142 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.462119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.464834 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.464812 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 14 15:53:47.464834 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.464829 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6qvcv\"" Jul 14 15:53:47.476274 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.476248 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx"] Jul 14 15:53:47.580659 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.580622 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl5hs\" (UniqueName: \"kubernetes.io/projected/9e620cf1-9864-4e84-8c69-3b4e2fa93583-kube-api-access-jl5hs\") pod \"kserve-controller-manager-d6f4b65b8-c8qtg\" (UID: \"9e620cf1-9864-4e84-8c69-3b4e2fa93583\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.580799 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.580684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b16e68-4d80-4cf2-b263-71eac1111b94-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-52rmx\" (UID: \"81b16e68-4d80-4cf2-b263-71eac1111b94\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.580799 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.580735 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e620cf1-9864-4e84-8c69-3b4e2fa93583-cert\") pod \"kserve-controller-manager-d6f4b65b8-c8qtg\" (UID: \"9e620cf1-9864-4e84-8c69-3b4e2fa93583\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.580885 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.580828 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh747\" (UniqueName: \"kubernetes.io/projected/81b16e68-4d80-4cf2-b263-71eac1111b94-kube-api-access-rh747\") pod \"llmisvc-controller-manager-777c6bd7d5-52rmx\" (UID: \"81b16e68-4d80-4cf2-b263-71eac1111b94\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.681722 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.681632 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b16e68-4d80-4cf2-b263-71eac1111b94-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-52rmx\" (UID: \"81b16e68-4d80-4cf2-b263-71eac1111b94\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.681722 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.681692 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e620cf1-9864-4e84-8c69-3b4e2fa93583-cert\") pod \"kserve-controller-manager-d6f4b65b8-c8qtg\" (UID: \"9e620cf1-9864-4e84-8c69-3b4e2fa93583\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.681947 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.681765 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh747\" (UniqueName: \"kubernetes.io/projected/81b16e68-4d80-4cf2-b263-71eac1111b94-kube-api-access-rh747\") pod \"llmisvc-controller-manager-777c6bd7d5-52rmx\" (UID: \"81b16e68-4d80-4cf2-b263-71eac1111b94\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.681947 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.681793 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl5hs\" (UniqueName: \"kubernetes.io/projected/9e620cf1-9864-4e84-8c69-3b4e2fa93583-kube-api-access-jl5hs\") pod \"kserve-controller-manager-d6f4b65b8-c8qtg\" (UID: \"9e620cf1-9864-4e84-8c69-3b4e2fa93583\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.684268 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.684243 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e620cf1-9864-4e84-8c69-3b4e2fa93583-cert\") pod \"kserve-controller-manager-d6f4b65b8-c8qtg\" (UID: \"9e620cf1-9864-4e84-8c69-3b4e2fa93583\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.684369 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.684287 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b16e68-4d80-4cf2-b263-71eac1111b94-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-52rmx\" (UID: \"81b16e68-4d80-4cf2-b263-71eac1111b94\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.705095 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.705065 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh747\" (UniqueName: \"kubernetes.io/projected/81b16e68-4d80-4cf2-b263-71eac1111b94-kube-api-access-rh747\") pod \"llmisvc-controller-manager-777c6bd7d5-52rmx\" (UID: \"81b16e68-4d80-4cf2-b263-71eac1111b94\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.705545 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.705526 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl5hs\" (UniqueName: \"kubernetes.io/projected/9e620cf1-9864-4e84-8c69-3b4e2fa93583-kube-api-access-jl5hs\") pod \"kserve-controller-manager-d6f4b65b8-c8qtg\" (UID: \"9e620cf1-9864-4e84-8c69-3b4e2fa93583\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.742141 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.742104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:47.772437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.772403 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:47.910884 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.910845 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg"] Jul 14 15:53:47.914343 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:47.914315 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e620cf1_9864_4e84_8c69_3b4e2fa93583.slice/crio-61d35336e02901f3e1e95093a76ddb1768ffe588c394802d3e0d3401088e78f8 WatchSource:0}: Error finding container 61d35336e02901f3e1e95093a76ddb1768ffe588c394802d3e0d3401088e78f8: Status 404 returned error can't find the container with id 61d35336e02901f3e1e95093a76ddb1768ffe588c394802d3e0d3401088e78f8 Jul 14 15:53:47.941439 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:47.941352 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx"] Jul 14 15:53:47.945338 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:53:47.945306 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81b16e68_4d80_4cf2_b263_71eac1111b94.slice/crio-58ea746792102c5c1722314762ffc4238c970e49dabe400490eabb4607dcc03f WatchSource:0}: Error finding container 58ea746792102c5c1722314762ffc4238c970e49dabe400490eabb4607dcc03f: Status 404 returned error can't find the container with id 58ea746792102c5c1722314762ffc4238c970e49dabe400490eabb4607dcc03f Jul 14 15:53:48.782868 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:48.782832 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" event={"ID":"81b16e68-4d80-4cf2-b263-71eac1111b94","Type":"ContainerStarted","Data":"58ea746792102c5c1722314762ffc4238c970e49dabe400490eabb4607dcc03f"} Jul 14 15:53:48.784103 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:48.784071 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" event={"ID":"9e620cf1-9864-4e84-8c69-3b4e2fa93583","Type":"ContainerStarted","Data":"61d35336e02901f3e1e95093a76ddb1768ffe588c394802d3e0d3401088e78f8"} Jul 14 15:53:49.979531 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:49.979497 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx"] Jul 14 15:53:49.979991 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:49.979716 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" podUID="2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" containerName="discovery" containerID="cri-o://b21ddeeaa243dc1e2ef4f74fe99a86daaa20e8119b85386d0be236f49846d466" gracePeriod=30 Jul 14 15:53:50.797851 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:50.797811 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" containerID="b21ddeeaa243dc1e2ef4f74fe99a86daaa20e8119b85386d0be236f49846d466" exitCode=0 Jul 14 15:53:50.798014 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:50.797886 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" event={"ID":"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb","Type":"ContainerDied","Data":"b21ddeeaa243dc1e2ef4f74fe99a86daaa20e8119b85386d0be236f49846d466"} Jul 14 15:53:51.183190 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.183164 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:51.323098 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.322999 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsq4z\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-kube-api-access-hsq4z\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323155 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-token\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323183 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-local-certs\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323208 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-kubeconfig\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323233 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-dns-cert\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323475 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323251 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-ca-configmap\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323475 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323310 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-cacerts\") pod \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\" (UID: \"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb\") " Jul 14 15:53:51.323799 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.323738 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 15:53:51.330763 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.330730 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-local-certs" (OuterVolumeSpecName: "local-certs") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 15:53:51.330874 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.330737 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:53:51.330874 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.330820 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-kube-api-access-hsq4z" (OuterVolumeSpecName: "kube-api-access-hsq4z") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "kube-api-access-hsq4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:53:51.330874 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.330859 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:53:51.331012 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.330941 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:53:51.331012 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.330963 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-cacerts" (OuterVolumeSpecName: "cacerts") pod "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" (UID: "2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424657 2573 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-cacerts\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424687 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsq4z\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-kube-api-access-hsq4z\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424700 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-token\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424709 2573 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-local-certs\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424717 2573 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-kubeconfig\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424725 2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-dns-cert\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.424738 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.424733 2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb-istio-csr-ca-configmap\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:53:51.804166 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.804084 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" Jul 14 15:53:51.804329 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.804082 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx" event={"ID":"2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb","Type":"ContainerDied","Data":"d392cdf873f73e63dab61904120ebf13f361ad9d1fc5a1500068d77af7dc714f"} Jul 14 15:53:51.804329 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.804288 2573 scope.go:117] "RemoveContainer" containerID="b21ddeeaa243dc1e2ef4f74fe99a86daaa20e8119b85386d0be236f49846d466" Jul 14 15:53:51.805910 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.805870 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" event={"ID":"81b16e68-4d80-4cf2-b263-71eac1111b94","Type":"ContainerStarted","Data":"ee1628850987d5b15e62f3105bbf361d01a9ad84fc301397959a179acb7439cf"} Jul 14 15:53:51.806085 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.806063 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:53:51.807712 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.807686 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" event={"ID":"9e620cf1-9864-4e84-8c69-3b4e2fa93583","Type":"ContainerStarted","Data":"6af4cdcacb58721b8a6bc3e850e578559681c7cf9ecaa488e523e94b328e0a9c"} Jul 14 15:53:51.807848 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.807832 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:53:51.844527 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.844474 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" podStartSLOduration=1.573903577 podStartE2EDuration="4.844458005s" totalImagesPullingTime="3.270554428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:51.843236373 +0000 UTC m=+427.229323760" watchObservedRunningTime="2026-07-14 15:53:51.844458005 +0000 UTC m=+427.230545383" Jul 14 15:53:51.909805 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.909744 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" podStartSLOduration=1.667473963 podStartE2EDuration="4.909726774s" totalImagesPullingTime="3.242252811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:51.908289442 +0000 UTC m=+427.294376830" watchObservedRunningTime="2026-07-14 15:53:51.909726774 +0000 UTC m=+427.295814161" Jul 14 15:53:51.931408 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.931359 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx"] Jul 14 15:53:51.932765 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:51.932744 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qrmrx"] Jul 14 15:53:52.758751 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:52.758721 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-n88b6" Jul 14 15:53:52.760891 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:52.760871 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-g52jn" Jul 14 15:53:53.098100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:53:53.098017 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" path="/var/lib/kubelet/pods/2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb/volumes" Jul 14 15:54:22.815494 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:22.815456 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-52rmx" Jul 14 15:54:22.818438 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:22.818419 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c8qtg" Jul 14 15:54:35.917762 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.917724 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhjz"] Jul 14 15:54:35.918148 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.918051 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" containerName="discovery" Jul 14 15:54:35.918148 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.918064 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" containerName="discovery" Jul 14 15:54:35.918148 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.918127 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0f0a46-5e36-4bbd-8f47-da8e6e949ceb" containerName="discovery" Jul 14 15:54:35.920778 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.920761 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:35.923758 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.923734 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 15:54:35.923905 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.923797 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kcg5g\"" Jul 14 15:54:35.924324 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.924308 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 15:54:35.940924 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.940894 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhjz"] Jul 14 15:54:35.978993 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:35.978960 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqsrk\" (UniqueName: \"kubernetes.io/projected/c34d65b5-899c-4760-9008-58565a56b9fd-kube-api-access-zqsrk\") pod \"kuadrant-operator-catalog-pjhjz\" (UID: \"c34d65b5-899c-4760-9008-58565a56b9fd\") " pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:36.080062 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.080000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqsrk\" (UniqueName: \"kubernetes.io/projected/c34d65b5-899c-4760-9008-58565a56b9fd-kube-api-access-zqsrk\") pod \"kuadrant-operator-catalog-pjhjz\" (UID: \"c34d65b5-899c-4760-9008-58565a56b9fd\") " pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:36.089772 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.089747 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqsrk\" (UniqueName: \"kubernetes.io/projected/c34d65b5-899c-4760-9008-58565a56b9fd-kube-api-access-zqsrk\") pod \"kuadrant-operator-catalog-pjhjz\" (UID: \"c34d65b5-899c-4760-9008-58565a56b9fd\") " pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:36.231230 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.231143 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:36.261978 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.261937 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhjz"] Jul 14 15:54:36.367564 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.367538 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhjz"] Jul 14 15:54:36.370301 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:54:36.370265 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc34d65b5_899c_4760_9008_58565a56b9fd.slice/crio-b41d0c8b1ad91a7c9cf5d058b7575ba5766a68e7da12b15278319b12a3926620 WatchSource:0}: Error finding container b41d0c8b1ad91a7c9cf5d058b7575ba5766a68e7da12b15278319b12a3926620: Status 404 returned error can't find the container with id b41d0c8b1ad91a7c9cf5d058b7575ba5766a68e7da12b15278319b12a3926620 Jul 14 15:54:36.480946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.480913 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6slqv"] Jul 14 15:54:36.486940 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.486184 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:36.497524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.497498 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6slqv"] Jul 14 15:54:36.588800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.588764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnwr4\" (UniqueName: \"kubernetes.io/projected/b4b50c86-cd5f-4bef-90d5-d6155056a1dc-kube-api-access-hnwr4\") pod \"kuadrant-operator-catalog-6slqv\" (UID: \"b4b50c86-cd5f-4bef-90d5-d6155056a1dc\") " pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:36.689600 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.689571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnwr4\" (UniqueName: \"kubernetes.io/projected/b4b50c86-cd5f-4bef-90d5-d6155056a1dc-kube-api-access-hnwr4\") pod \"kuadrant-operator-catalog-6slqv\" (UID: \"b4b50c86-cd5f-4bef-90d5-d6155056a1dc\") " pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:36.702965 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.702934 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnwr4\" (UniqueName: \"kubernetes.io/projected/b4b50c86-cd5f-4bef-90d5-d6155056a1dc-kube-api-access-hnwr4\") pod \"kuadrant-operator-catalog-6slqv\" (UID: \"b4b50c86-cd5f-4bef-90d5-d6155056a1dc\") " pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:36.796643 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.796556 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:36.944603 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.944578 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6slqv"] Jul 14 15:54:36.946990 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:54:36.946959 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b50c86_cd5f_4bef_90d5_d6155056a1dc.slice/crio-8fa10eb14f112c946a4ec271aa31213a4f1b364beb66f0ba4c1d782a3de24c0e WatchSource:0}: Error finding container 8fa10eb14f112c946a4ec271aa31213a4f1b364beb66f0ba4c1d782a3de24c0e: Status 404 returned error can't find the container with id 8fa10eb14f112c946a4ec271aa31213a4f1b364beb66f0ba4c1d782a3de24c0e Jul 14 15:54:36.994721 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.994684 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" event={"ID":"b4b50c86-cd5f-4bef-90d5-d6155056a1dc","Type":"ContainerStarted","Data":"8fa10eb14f112c946a4ec271aa31213a4f1b364beb66f0ba4c1d782a3de24c0e"} Jul 14 15:54:36.995948 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:36.995918 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" event={"ID":"c34d65b5-899c-4760-9008-58565a56b9fd","Type":"ContainerStarted","Data":"b41d0c8b1ad91a7c9cf5d058b7575ba5766a68e7da12b15278319b12a3926620"} Jul 14 15:54:39.007868 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.007831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" event={"ID":"b4b50c86-cd5f-4bef-90d5-d6155056a1dc","Type":"ContainerStarted","Data":"2b20e5bed5c676a45dc4cede1ca2180db4d64b1beb838db35c4e77e57c5bf2da"} Jul 14 15:54:39.009225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.009200 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" event={"ID":"c34d65b5-899c-4760-9008-58565a56b9fd","Type":"ContainerStarted","Data":"9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab"} Jul 14 15:54:39.009351 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.009300 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" podUID="c34d65b5-899c-4760-9008-58565a56b9fd" containerName="registry-server" containerID="cri-o://9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab" gracePeriod=2 Jul 14 15:54:39.028752 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.028713 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" podStartSLOduration=1.515505903 podStartE2EDuration="3.028698677s" totalImagesPullingTime="1.513192774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:54:39.028174581 +0000 UTC m=+474.414261970" watchObservedRunningTime="2026-07-14 15:54:39.028698677 +0000 UTC m=+474.414786055" Jul 14 15:54:39.045692 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.045652 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" podStartSLOduration=1.955845102 podStartE2EDuration="4.045637807s" totalImagesPullingTime="2.089792705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:54:39.045161602 +0000 UTC m=+474.431249000" watchObservedRunningTime="2026-07-14 15:54:39.045637807 +0000 UTC m=+474.431725194" Jul 14 15:54:39.251817 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.251788 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:39.312073 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.311961 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqsrk\" (UniqueName: \"kubernetes.io/projected/c34d65b5-899c-4760-9008-58565a56b9fd-kube-api-access-zqsrk\") pod \"c34d65b5-899c-4760-9008-58565a56b9fd\" (UID: \"c34d65b5-899c-4760-9008-58565a56b9fd\") " Jul 14 15:54:39.314255 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.314231 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c34d65b5-899c-4760-9008-58565a56b9fd-kube-api-access-zqsrk" (OuterVolumeSpecName: "kube-api-access-zqsrk") pod "c34d65b5-899c-4760-9008-58565a56b9fd" (UID: "c34d65b5-899c-4760-9008-58565a56b9fd"). InnerVolumeSpecName "kube-api-access-zqsrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:54:39.412561 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:39.412520 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqsrk\" (UniqueName: \"kubernetes.io/projected/c34d65b5-899c-4760-9008-58565a56b9fd-kube-api-access-zqsrk\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:54:40.014800 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.014767 2573 generic.go:356] "Generic (PLEG): container finished" podID="c34d65b5-899c-4760-9008-58565a56b9fd" containerID="9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab" exitCode=0 Jul 14 15:54:40.015213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.014812 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" Jul 14 15:54:40.015213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.014853 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" event={"ID":"c34d65b5-899c-4760-9008-58565a56b9fd","Type":"ContainerDied","Data":"9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab"} Jul 14 15:54:40.015213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.014900 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjhjz" event={"ID":"c34d65b5-899c-4760-9008-58565a56b9fd","Type":"ContainerDied","Data":"b41d0c8b1ad91a7c9cf5d058b7575ba5766a68e7da12b15278319b12a3926620"} Jul 14 15:54:40.015213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.014920 2573 scope.go:117] "RemoveContainer" containerID="9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab" Jul 14 15:54:40.024205 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.024186 2573 scope.go:117] "RemoveContainer" containerID="9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab" Jul 14 15:54:40.024505 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:54:40.024487 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab\": container with ID starting with 9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab not found: ID does not exist" containerID="9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab" Jul 14 15:54:40.024552 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.024515 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab"} err="failed to get container status \"9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab\": rpc error: code = NotFound desc = could not find container \"9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab\": container with ID starting with 9c55fd99677de021068b86476d45bb73354b7daf491299522fe35a791996bbab not found: ID does not exist" Jul 14 15:54:40.047233 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.047203 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhjz"] Jul 14 15:54:40.050833 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:40.050812 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhjz"] Jul 14 15:54:41.097884 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:41.097852 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c34d65b5-899c-4760-9008-58565a56b9fd" path="/var/lib/kubelet/pods/c34d65b5-899c-4760-9008-58565a56b9fd/volumes" Jul 14 15:54:46.796939 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:46.796896 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:46.797371 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:46.796988 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:46.820287 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:46.820260 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:54:47.071881 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:54:47.071806 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6slqv" Jul 14 15:55:04.569303 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.569259 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8"] Jul 14 15:55:04.569882 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.569688 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c34d65b5-899c-4760-9008-58565a56b9fd" containerName="registry-server" Jul 14 15:55:04.569882 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.569707 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34d65b5-899c-4760-9008-58565a56b9fd" containerName="registry-server" Jul 14 15:55:04.569882 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.569820 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c34d65b5-899c-4760-9008-58565a56b9fd" containerName="registry-server" Jul 14 15:55:04.574477 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.574455 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.577472 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.577453 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zqvcz\"" Jul 14 15:55:04.588561 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.588536 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8"] Jul 14 15:55:04.626254 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.626220 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.626437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.626264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dxdg\" (UniqueName: \"kubernetes.io/projected/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-kube-api-access-5dxdg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.726689 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.726645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.726862 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.726706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dxdg\" (UniqueName: \"kubernetes.io/projected/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-kube-api-access-5dxdg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.727078 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.727024 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.741324 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.741292 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dxdg\" (UniqueName: \"kubernetes.io/projected/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-kube-api-access-5dxdg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:04.884499 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:04.884469 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:05.043064 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:05.043014 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8"] Jul 14 15:55:05.046016 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:55:05.045989 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fb0fcc_c1a9_4231_8706_207ce2cc523e.slice/crio-dacef8b06d3feab9556183bf9add1dd274e1cbca13d635e47af25382ebbf6de8 WatchSource:0}: Error finding container dacef8b06d3feab9556183bf9add1dd274e1cbca13d635e47af25382ebbf6de8: Status 404 returned error can't find the container with id dacef8b06d3feab9556183bf9add1dd274e1cbca13d635e47af25382ebbf6de8 Jul 14 15:55:05.126001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:05.125959 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" event={"ID":"e4fb0fcc-c1a9-4231-8706-207ce2cc523e","Type":"ContainerStarted","Data":"dacef8b06d3feab9556183bf9add1dd274e1cbca13d635e47af25382ebbf6de8"} Jul 14 15:55:07.284885 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.284848 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8p26s"] Jul 14 15:55:07.288408 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.288391 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:07.293141 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.293120 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-g28mm\"" Jul 14 15:55:07.302410 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.302384 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8p26s"] Jul 14 15:55:07.345002 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.344965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvfx\" (UniqueName: \"kubernetes.io/projected/67a6c273-40ab-41db-8cba-ccbe115c5a89-kube-api-access-9nvfx\") pod \"authorino-operator-75cc8d5cb7-8p26s\" (UID: \"67a6c273-40ab-41db-8cba-ccbe115c5a89\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:07.445852 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.445814 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvfx\" (UniqueName: \"kubernetes.io/projected/67a6c273-40ab-41db-8cba-ccbe115c5a89-kube-api-access-9nvfx\") pod \"authorino-operator-75cc8d5cb7-8p26s\" (UID: \"67a6c273-40ab-41db-8cba-ccbe115c5a89\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:07.466884 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.466843 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvfx\" (UniqueName: \"kubernetes.io/projected/67a6c273-40ab-41db-8cba-ccbe115c5a89-kube-api-access-9nvfx\") pod \"authorino-operator-75cc8d5cb7-8p26s\" (UID: \"67a6c273-40ab-41db-8cba-ccbe115c5a89\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:07.599432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.599339 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:07.747012 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:07.746789 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8p26s"] Jul 14 15:55:07.751754 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:55:07.751677 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a6c273_40ab_41db_8cba_ccbe115c5a89.slice/crio-80c0a6407ec1f6c522615f9ad7f7aa258a182d4516596dd1c6936b1bd76cd594 WatchSource:0}: Error finding container 80c0a6407ec1f6c522615f9ad7f7aa258a182d4516596dd1c6936b1bd76cd594: Status 404 returned error can't find the container with id 80c0a6407ec1f6c522615f9ad7f7aa258a182d4516596dd1c6936b1bd76cd594 Jul 14 15:55:08.142433 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:08.142396 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" event={"ID":"67a6c273-40ab-41db-8cba-ccbe115c5a89","Type":"ContainerStarted","Data":"80c0a6407ec1f6c522615f9ad7f7aa258a182d4516596dd1c6936b1bd76cd594"} Jul 14 15:55:09.425216 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.425183 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt"] Jul 14 15:55:09.434286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.434251 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:09.437476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.437332 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hh6v7\"" Jul 14 15:55:09.438315 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.438262 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt"] Jul 14 15:55:09.460529 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.460482 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzcdb\" (UniqueName: \"kubernetes.io/projected/e4ecb4da-be69-4447-a373-24bac084fb6c-kube-api-access-jzcdb\") pod \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" (UID: \"e4ecb4da-be69-4447-a373-24bac084fb6c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:09.561840 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.561808 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzcdb\" (UniqueName: \"kubernetes.io/projected/e4ecb4da-be69-4447-a373-24bac084fb6c-kube-api-access-jzcdb\") pod \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" (UID: \"e4ecb4da-be69-4447-a373-24bac084fb6c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:09.573235 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.573193 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzcdb\" (UniqueName: \"kubernetes.io/projected/e4ecb4da-be69-4447-a373-24bac084fb6c-kube-api-access-jzcdb\") pod \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" (UID: \"e4ecb4da-be69-4447-a373-24bac084fb6c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:09.749898 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:09.749805 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:11.456350 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:11.456322 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt"] Jul 14 15:55:11.458903 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:55:11.458874 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ecb4da_be69_4447_a373_24bac084fb6c.slice/crio-ddcf3b6b3fd57fa838ee98f77134c84c438b0ed4c8a09d2d2e031123c826640e WatchSource:0}: Error finding container ddcf3b6b3fd57fa838ee98f77134c84c438b0ed4c8a09d2d2e031123c826640e: Status 404 returned error can't find the container with id ddcf3b6b3fd57fa838ee98f77134c84c438b0ed4c8a09d2d2e031123c826640e Jul 14 15:55:12.167416 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.167371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" event={"ID":"e4fb0fcc-c1a9-4231-8706-207ce2cc523e","Type":"ContainerStarted","Data":"082736ace5dc34eebf0af487d4597aab98d06dacdce27d15687839f4669a808b"} Jul 14 15:55:12.167590 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.167454 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:12.169189 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.169157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" event={"ID":"e4ecb4da-be69-4447-a373-24bac084fb6c","Type":"ContainerStarted","Data":"ddcf3b6b3fd57fa838ee98f77134c84c438b0ed4c8a09d2d2e031123c826640e"} Jul 14 15:55:12.170607 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.170575 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" event={"ID":"67a6c273-40ab-41db-8cba-ccbe115c5a89","Type":"ContainerStarted","Data":"c83de02104f168dab929355eff76830e357dbbeb86f086179cef5c499b756da2"} Jul 14 15:55:12.170716 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.170685 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:12.190470 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.190410 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" podStartSLOduration=1.910090409 podStartE2EDuration="8.190390289s" totalImagesPullingTime="6.28029988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:12.18931174 +0000 UTC m=+507.575399127" watchObservedRunningTime="2026-07-14 15:55:12.190390289 +0000 UTC m=+507.576477752" Jul 14 15:55:12.208433 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:12.208385 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" podStartSLOduration=1.6406164699999999 podStartE2EDuration="5.208367323s" totalImagesPullingTime="3.567750853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:12.207184912 +0000 UTC m=+507.593272301" watchObservedRunningTime="2026-07-14 15:55:12.208367323 +0000 UTC m=+507.594454710" Jul 14 15:55:14.181724 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:14.181685 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" event={"ID":"e4ecb4da-be69-4447-a373-24bac084fb6c","Type":"ContainerStarted","Data":"59f1a9924ee6d46d68860b443fdba736cd6028b3b759fd3b7a49694ae3e8d2f7"} Jul 14 15:55:14.182141 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:14.181817 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:14.203911 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:14.203860 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" podStartSLOduration=3.337293685 podStartE2EDuration="5.203844387s" totalImagesPullingTime="1.866550702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:14.202902635 +0000 UTC m=+509.588990023" watchObservedRunningTime="2026-07-14 15:55:14.203844387 +0000 UTC m=+509.589931775" Jul 14 15:55:23.177720 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:23.177687 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8p26s" Jul 14 15:55:23.178186 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:23.177773 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:24.888350 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:24.888314 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685"] Jul 14 15:55:24.891744 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:24.891726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:24.905401 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:24.905374 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685"] Jul 14 15:55:24.984758 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:24.984720 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a71005b1-6091-4413-8704-7db3818af90d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rf685\" (UID: \"a71005b1-6091-4413-8704-7db3818af90d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:24.984934 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:24.984815 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x95s4\" (UniqueName: \"kubernetes.io/projected/a71005b1-6091-4413-8704-7db3818af90d-kube-api-access-x95s4\") pod \"kuadrant-operator-controller-manager-86878c769f-rf685\" (UID: \"a71005b1-6091-4413-8704-7db3818af90d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.007532 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.007501 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8"] Jul 14 15:55:25.007712 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.007687 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" containerName="manager" containerID="cri-o://082736ace5dc34eebf0af487d4597aab98d06dacdce27d15687839f4669a808b" gracePeriod=2 Jul 14 15:55:25.019864 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.019834 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8"] Jul 14 15:55:25.031424 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.031396 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685"] Jul 14 15:55:25.031660 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.031641 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-x95s4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" podUID="a71005b1-6091-4413-8704-7db3818af90d" Jul 14 15:55:25.043828 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.043795 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685"] Jul 14 15:55:25.046169 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.046143 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9"] Jul 14 15:55:25.046631 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.046606 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" containerName="manager" Jul 14 15:55:25.046631 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.046632 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" containerName="manager" Jul 14 15:55:25.046791 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.046735 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" containerName="manager" Jul 14 15:55:25.049877 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.049854 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.052369 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.052334 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:25.059121 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.059101 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt"] Jul 14 15:55:25.059329 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.059304 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" containerName="manager" containerID="cri-o://59f1a9924ee6d46d68860b443fdba736cd6028b3b759fd3b7a49694ae3e8d2f7" gracePeriod=2 Jul 14 15:55:25.061746 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.061697 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:25.062214 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.062191 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq"] Jul 14 15:55:25.065717 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.065694 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9"] Jul 14 15:55:25.065814 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.065799 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.070178 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.070159 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt"] Jul 14 15:55:25.085326 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.085298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a71005b1-6091-4413-8704-7db3818af90d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rf685\" (UID: \"a71005b1-6091-4413-8704-7db3818af90d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.085437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.085345 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0320c274-2703-46a4-807f-60a90d0f41ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flxb9\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.085437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.085373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2jtv\" (UniqueName: \"kubernetes.io/projected/0320c274-2703-46a4-807f-60a90d0f41ec-kube-api-access-b2jtv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flxb9\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.085541 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.085527 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x95s4\" (UniqueName: \"kubernetes.io/projected/a71005b1-6091-4413-8704-7db3818af90d-kube-api-access-x95s4\") pod \"kuadrant-operator-controller-manager-86878c769f-rf685\" (UID: \"a71005b1-6091-4413-8704-7db3818af90d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.085768 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.085746 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a71005b1-6091-4413-8704-7db3818af90d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rf685\" (UID: \"a71005b1-6091-4413-8704-7db3818af90d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.089608 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.089564 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:25.092086 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.091386 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:25.092759 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.092623 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq"] Jul 14 15:55:25.092866 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.092625 2573 projected.go:196] Error preparing data for projected volume kube-api-access-x95s4 for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-rf685" not found Jul 14 15:55:25.092921 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.092906 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a71005b1-6091-4413-8704-7db3818af90d-kube-api-access-x95s4 podName:a71005b1-6091-4413-8704-7db3818af90d nodeName:}" failed. No retries permitted until 2026-07-14 15:55:25.592871793 +0000 UTC m=+520.978959264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x95s4" (UniqueName: "kubernetes.io/projected/a71005b1-6091-4413-8704-7db3818af90d-kube-api-access-x95s4") pod "kuadrant-operator-controller-manager-86878c769f-rf685" (UID: "a71005b1-6091-4413-8704-7db3818af90d") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-rf685" not found Jul 14 15:55:25.186482 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.186451 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdht6\" (UniqueName: \"kubernetes.io/projected/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-kube-api-access-gdht6\") pod \"kuadrant-operator-controller-manager-86878c769f-qpvtq\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.186482 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.186492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qpvtq\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.186735 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.186718 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0320c274-2703-46a4-807f-60a90d0f41ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flxb9\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.186791 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.186743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jtv\" (UniqueName: \"kubernetes.io/projected/0320c274-2703-46a4-807f-60a90d0f41ec-kube-api-access-b2jtv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flxb9\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.187140 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.187117 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0320c274-2703-46a4-807f-60a90d0f41ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flxb9\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.201427 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.201398 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2jtv\" (UniqueName: \"kubernetes.io/projected/0320c274-2703-46a4-807f-60a90d0f41ec-kube-api-access-b2jtv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-flxb9\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.233360 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.233325 2573 generic.go:356] "Generic (PLEG): container finished" podID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" containerID="082736ace5dc34eebf0af487d4597aab98d06dacdce27d15687839f4669a808b" exitCode=0 Jul 14 15:55:25.235264 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.235242 2573 generic.go:356] "Generic (PLEG): container finished" podID="e4ecb4da-be69-4447-a373-24bac084fb6c" containerID="59f1a9924ee6d46d68860b443fdba736cd6028b3b759fd3b7a49694ae3e8d2f7" exitCode=0 Jul 14 15:55:25.235595 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.235415 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.238844 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.238640 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-rf685\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="a71005b1-6091-4413-8704-7db3818af90d" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.244109 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.244088 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.247944 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.247909 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-rf685\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="a71005b1-6091-4413-8704-7db3818af90d" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.287683 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.287648 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a71005b1-6091-4413-8704-7db3818af90d-extensions-socket-volume\") pod \"a71005b1-6091-4413-8704-7db3818af90d\" (UID: \"a71005b1-6091-4413-8704-7db3818af90d\") " Jul 14 15:55:25.287919 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.287877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdht6\" (UniqueName: \"kubernetes.io/projected/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-kube-api-access-gdht6\") pod \"kuadrant-operator-controller-manager-86878c769f-qpvtq\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.287919 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.287902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qpvtq\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.288012 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.287938 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x95s4\" (UniqueName: \"kubernetes.io/projected/a71005b1-6091-4413-8704-7db3818af90d-kube-api-access-x95s4\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:25.288012 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.287983 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a71005b1-6091-4413-8704-7db3818af90d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a71005b1-6091-4413-8704-7db3818af90d" (UID: "a71005b1-6091-4413-8704-7db3818af90d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 15:55:25.288273 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.288237 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qpvtq\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.296923 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.296903 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:25.299258 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.299233 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:25.299836 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.299821 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:25.302758 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.302736 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-rf685\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="a71005b1-6091-4413-8704-7db3818af90d" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.306542 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.306520 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:25.310641 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.310622 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:25.314380 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:25.314348 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-rf685\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="a71005b1-6091-4413-8704-7db3818af90d" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:25.314479 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.314468 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdht6\" (UniqueName: \"kubernetes.io/projected/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-kube-api-access-gdht6\") pod \"kuadrant-operator-controller-manager-86878c769f-qpvtq\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.388574 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.388534 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-extensions-socket-volume\") pod \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " Jul 14 15:55:25.388736 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.388613 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dxdg\" (UniqueName: \"kubernetes.io/projected/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-kube-api-access-5dxdg\") pod \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\" (UID: \"e4fb0fcc-c1a9-4231-8706-207ce2cc523e\") " Jul 14 15:55:25.388736 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.388660 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzcdb\" (UniqueName: \"kubernetes.io/projected/e4ecb4da-be69-4447-a373-24bac084fb6c-kube-api-access-jzcdb\") pod \"e4ecb4da-be69-4447-a373-24bac084fb6c\" (UID: \"e4ecb4da-be69-4447-a373-24bac084fb6c\") " Jul 14 15:55:25.388880 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.388767 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a71005b1-6091-4413-8704-7db3818af90d-extensions-socket-volume\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:25.389105 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.389075 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e4fb0fcc-c1a9-4231-8706-207ce2cc523e" (UID: "e4fb0fcc-c1a9-4231-8706-207ce2cc523e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 15:55:25.390789 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.390767 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-kube-api-access-5dxdg" (OuterVolumeSpecName: "kube-api-access-5dxdg") pod "e4fb0fcc-c1a9-4231-8706-207ce2cc523e" (UID: "e4fb0fcc-c1a9-4231-8706-207ce2cc523e"). InnerVolumeSpecName "kube-api-access-5dxdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:55:25.390789 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.390774 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ecb4da-be69-4447-a373-24bac084fb6c-kube-api-access-jzcdb" (OuterVolumeSpecName: "kube-api-access-jzcdb") pod "e4ecb4da-be69-4447-a373-24bac084fb6c" (UID: "e4ecb4da-be69-4447-a373-24bac084fb6c"). InnerVolumeSpecName "kube-api-access-jzcdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:55:25.443169 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.443135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:25.449329 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.449307 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:25.489985 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.489950 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dxdg\" (UniqueName: \"kubernetes.io/projected/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-kube-api-access-5dxdg\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:25.489985 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.489978 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzcdb\" (UniqueName: \"kubernetes.io/projected/e4ecb4da-be69-4447-a373-24bac084fb6c-kube-api-access-jzcdb\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:25.489985 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.489991 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4fb0fcc-c1a9-4231-8706-207ce2cc523e-extensions-socket-volume\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:25.595921 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.595891 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9"] Jul 14 15:55:25.599486 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:55:25.599461 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0320c274_2703_46a4_807f_60a90d0f41ec.slice/crio-ca3efa7a845452861160d02fcd10cc42dbb4f987bc5aa6be42fdcc08adcd21b7 WatchSource:0}: Error finding container ca3efa7a845452861160d02fcd10cc42dbb4f987bc5aa6be42fdcc08adcd21b7: Status 404 returned error can't find the container with id ca3efa7a845452861160d02fcd10cc42dbb4f987bc5aa6be42fdcc08adcd21b7 Jul 14 15:55:25.617175 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:25.617077 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq"] Jul 14 15:55:25.619290 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:55:25.619261 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ff41c7_ba7c_4e6f_a1b2_3d3eef77a4bd.slice/crio-7bf83fc63a9feb68af410eb963d6d89d842a80289cf8ee9230e7e188d1ca5ae6 WatchSource:0}: Error finding container 7bf83fc63a9feb68af410eb963d6d89d842a80289cf8ee9230e7e188d1ca5ae6: Status 404 returned error can't find the container with id 7bf83fc63a9feb68af410eb963d6d89d842a80289cf8ee9230e7e188d1ca5ae6 Jul 14 15:55:26.243320 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.243287 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" event={"ID":"0320c274-2703-46a4-807f-60a90d0f41ec","Type":"ContainerStarted","Data":"e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5"} Jul 14 15:55:26.243725 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.243349 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" event={"ID":"0320c274-2703-46a4-807f-60a90d0f41ec","Type":"ContainerStarted","Data":"ca3efa7a845452861160d02fcd10cc42dbb4f987bc5aa6be42fdcc08adcd21b7"} Jul 14 15:55:26.243725 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.243378 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:26.244721 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.244697 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:26.244846 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.244739 2573 scope.go:117] "RemoveContainer" containerID="082736ace5dc34eebf0af487d4597aab98d06dacdce27d15687839f4669a808b" Jul 14 15:55:26.245596 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.245564 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:26.246306 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.246277 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:26.247336 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.247302 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:26.248530 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.248503 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:26.248620 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.248554 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" event={"ID":"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd","Type":"ContainerStarted","Data":"0badff99b542d7b446d04742c7de60b9e87688d4c023b9a86b4629ba788a95fa"} Jul 14 15:55:26.248790 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.248773 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:26.249068 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.249023 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" event={"ID":"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd","Type":"ContainerStarted","Data":"7bf83fc63a9feb68af410eb963d6d89d842a80289cf8ee9230e7e188d1ca5ae6"} Jul 14 15:55:26.249209 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.248945 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-rf685\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="a71005b1-6091-4413-8704-7db3818af90d" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:26.254473 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.254452 2573 scope.go:117] "RemoveContainer" containerID="59f1a9924ee6d46d68860b443fdba736cd6028b3b759fd3b7a49694ae3e8d2f7" Jul 14 15:55:26.269552 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.269512 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" podStartSLOduration=1.269497964 podStartE2EDuration="1.269497964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:26.26879279 +0000 UTC m=+521.654880176" watchObservedRunningTime="2026-07-14 15:55:26.269497964 +0000 UTC m=+521.655585351" Jul 14 15:55:26.270962 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.270932 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tqpn8\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tqpn8" Jul 14 15:55:26.272629 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.272606 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-rf685\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="a71005b1-6091-4413-8704-7db3818af90d" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rf685" Jul 14 15:55:26.279299 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.279240 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71005b1_6091_4413_8704_7db3818af90d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ecb4da_be69_4447_a373_24bac084fb6c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fb0fcc_c1a9_4231_8706_207ce2cc523e.slice\": RecentStats: unable to find data in memory cache]" Jul 14 15:55:26.293297 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:26.293247 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" podStartSLOduration=1.293232737 podStartE2EDuration="1.293232737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:26.291876033 +0000 UTC m=+521.677963420" watchObservedRunningTime="2026-07-14 15:55:26.293232737 +0000 UTC m=+521.679320123" Jul 14 15:55:26.293760 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:26.293735 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gj4mt\" is forbidden: User \"system:node:ip-10-0-139-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-147.ec2.internal' and this object" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gj4mt" Jul 14 15:55:27.098494 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:27.098462 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71005b1-6091-4413-8704-7db3818af90d" path="/var/lib/kubelet/pods/a71005b1-6091-4413-8704-7db3818af90d/volumes" Jul 14 15:55:27.098685 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:27.098673 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" path="/var/lib/kubelet/pods/e4ecb4da-be69-4447-a373-24bac084fb6c/volumes" Jul 14 15:55:27.098969 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:27.098957 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4fb0fcc-c1a9-4231-8706-207ce2cc523e" path="/var/lib/kubelet/pods/e4fb0fcc-c1a9-4231-8706-207ce2cc523e/volumes" Jul 14 15:55:37.258165 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:37.258074 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:38.264576 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.264545 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 15:55:38.315583 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.315548 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9"] Jul 14 15:55:38.315768 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.315747 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" podUID="0320c274-2703-46a4-807f-60a90d0f41ec" containerName="manager" containerID="cri-o://e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5" gracePeriod=10 Jul 14 15:55:38.576369 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.576344 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:38.698065 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.698019 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2jtv\" (UniqueName: \"kubernetes.io/projected/0320c274-2703-46a4-807f-60a90d0f41ec-kube-api-access-b2jtv\") pod \"0320c274-2703-46a4-807f-60a90d0f41ec\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " Jul 14 15:55:38.698238 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.698091 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0320c274-2703-46a4-807f-60a90d0f41ec-extensions-socket-volume\") pod \"0320c274-2703-46a4-807f-60a90d0f41ec\" (UID: \"0320c274-2703-46a4-807f-60a90d0f41ec\") " Jul 14 15:55:38.698497 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.698472 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0320c274-2703-46a4-807f-60a90d0f41ec-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0320c274-2703-46a4-807f-60a90d0f41ec" (UID: "0320c274-2703-46a4-807f-60a90d0f41ec"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 15:55:38.700316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.700294 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0320c274-2703-46a4-807f-60a90d0f41ec-kube-api-access-b2jtv" (OuterVolumeSpecName: "kube-api-access-b2jtv") pod "0320c274-2703-46a4-807f-60a90d0f41ec" (UID: "0320c274-2703-46a4-807f-60a90d0f41ec"). InnerVolumeSpecName "kube-api-access-b2jtv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:55:38.799725 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.799638 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2jtv\" (UniqueName: \"kubernetes.io/projected/0320c274-2703-46a4-807f-60a90d0f41ec-kube-api-access-b2jtv\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:38.799725 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:38.799668 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0320c274-2703-46a4-807f-60a90d0f41ec-extensions-socket-volume\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:55:39.310029 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.309987 2573 generic.go:356] "Generic (PLEG): container finished" podID="0320c274-2703-46a4-807f-60a90d0f41ec" containerID="e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5" exitCode=0 Jul 14 15:55:39.310464 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.310060 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" Jul 14 15:55:39.310464 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.310077 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" event={"ID":"0320c274-2703-46a4-807f-60a90d0f41ec","Type":"ContainerDied","Data":"e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5"} Jul 14 15:55:39.310464 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.310131 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9" event={"ID":"0320c274-2703-46a4-807f-60a90d0f41ec","Type":"ContainerDied","Data":"ca3efa7a845452861160d02fcd10cc42dbb4f987bc5aa6be42fdcc08adcd21b7"} Jul 14 15:55:39.310464 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.310152 2573 scope.go:117] "RemoveContainer" containerID="e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5" Jul 14 15:55:39.318822 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.318653 2573 scope.go:117] "RemoveContainer" containerID="e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5" Jul 14 15:55:39.318933 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:55:39.318914 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5\": container with ID starting with e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5 not found: ID does not exist" containerID="e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5" Jul 14 15:55:39.318972 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.318941 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5"} err="failed to get container status \"e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5\": rpc error: code = NotFound desc = could not find container \"e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5\": container with ID starting with e683b001ef8ee1b088b07455cc9c6051b96eb5ea7c5348a55a907ce707f68fe5 not found: ID does not exist" Jul 14 15:55:39.329556 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.329524 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9"] Jul 14 15:55:39.335116 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:39.335095 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-flxb9"] Jul 14 15:55:41.098778 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:41.098740 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0320c274-2703-46a4-807f-60a90d0f41ec" path="/var/lib/kubelet/pods/0320c274-2703-46a4-807f-60a90d0f41ec/volumes" Jul 14 15:55:54.728209 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728179 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2"] Jul 14 15:55:54.728578 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728495 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0320c274-2703-46a4-807f-60a90d0f41ec" containerName="manager" Jul 14 15:55:54.728578 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728505 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0320c274-2703-46a4-807f-60a90d0f41ec" containerName="manager" Jul 14 15:55:54.728578 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728516 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" containerName="manager" Jul 14 15:55:54.728578 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728522 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" containerName="manager" Jul 14 15:55:54.728578 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728570 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ecb4da-be69-4447-a373-24bac084fb6c" containerName="manager" Jul 14 15:55:54.728578 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.728578 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0320c274-2703-46a4-807f-60a90d0f41ec" containerName="manager" Jul 14 15:55:54.732464 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.732439 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.734840 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.734817 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 15:55:54.734992 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.734855 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rvhrj\"" Jul 14 15:55:54.743838 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.743816 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2"] Jul 14 15:55:54.843561 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843521 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8864abe0-921f-41b4-91da-ac151bf2da49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.843765 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8864abe0-921f-41b4-91da-ac151bf2da49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.843765 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbfgq\" (UniqueName: \"kubernetes.io/projected/8864abe0-921f-41b4-91da-ac151bf2da49-kube-api-access-vbfgq\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.843765 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.843765 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.843765 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843765 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.844009 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.844009 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843817 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8864abe0-921f-41b4-91da-ac151bf2da49-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.844009 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.843834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.944979 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.944937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8864abe0-921f-41b4-91da-ac151bf2da49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.944986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8864abe0-921f-41b4-91da-ac151bf2da49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbfgq\" (UniqueName: \"kubernetes.io/projected/8864abe0-921f-41b4-91da-ac151bf2da49-kube-api-access-vbfgq\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945213 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945391 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945391 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945297 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945391 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945391 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8864abe0-921f-41b4-91da-ac151bf2da49-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945598 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945393 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.945651 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945614 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.946018 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945694 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.946018 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945827 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.946018 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.945834 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.946259 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.946135 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8864abe0-921f-41b4-91da-ac151bf2da49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.947818 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.947800 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8864abe0-921f-41b4-91da-ac151bf2da49-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.947890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.947856 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8864abe0-921f-41b4-91da-ac151bf2da49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.954572 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.954547 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbfgq\" (UniqueName: \"kubernetes.io/projected/8864abe0-921f-41b4-91da-ac151bf2da49-kube-api-access-vbfgq\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:54.954720 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:54.954649 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8864abe0-921f-41b4-91da-ac151bf2da49-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m2zw2\" (UID: \"8864abe0-921f-41b4-91da-ac151bf2da49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:55.043963 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:55.043868 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:55.189832 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:55.189800 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2"] Jul 14 15:55:55.194544 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:55:55.194507 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8864abe0_921f_41b4_91da_ac151bf2da49.slice/crio-39f8b2a1500cf8b6dc2e5fc024a5e55952e868503764b4e2919030ade4139492 WatchSource:0}: Error finding container 39f8b2a1500cf8b6dc2e5fc024a5e55952e868503764b4e2919030ade4139492: Status 404 returned error can't find the container with id 39f8b2a1500cf8b6dc2e5fc024a5e55952e868503764b4e2919030ade4139492 Jul 14 15:55:55.383805 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:55.383721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" event={"ID":"8864abe0-921f-41b4-91da-ac151bf2da49","Type":"ContainerStarted","Data":"39f8b2a1500cf8b6dc2e5fc024a5e55952e868503764b4e2919030ade4139492"} Jul 14 15:55:57.700696 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:57.700657 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:55:57.700966 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:57.700742 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:55:57.700966 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:57.700776 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:55:58.399766 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:58.399735 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" event={"ID":"8864abe0-921f-41b4-91da-ac151bf2da49","Type":"ContainerStarted","Data":"0bbd73f9124dc0bd7922d1c1e964c2a7b084fb3deb80683c7e30e748d4fa556c"} Jul 14 15:55:58.421237 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:58.421179 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" podStartSLOduration=1.917508868 podStartE2EDuration="4.421163204s" totalImagesPullingTime="2.503654336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:58.420767572 +0000 UTC m=+553.806854974" watchObservedRunningTime="2026-07-14 15:55:58.421163204 +0000 UTC m=+553.807250591" Jul 14 15:55:59.044548 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:59.044507 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:59.049315 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:59.049289 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:59.405184 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:59.405151 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:55:59.405995 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:55:59.405977 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:56:00.184972 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.184935 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:00.187446 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.187423 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.189806 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.189780 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 15:56:00.189939 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.189863 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wt668\"" Jul 14 15:56:00.196378 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.196355 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:00.279945 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.279903 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:00.292635 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.292602 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-config-file\") pod \"limitador-limitador-64c94fd64f-k2w57\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.292787 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.292652 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf4mg\" (UniqueName: \"kubernetes.io/projected/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-kube-api-access-jf4mg\") pod \"limitador-limitador-64c94fd64f-k2w57\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.300831 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.300793 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-tfxtb"] Jul 14 15:56:00.303087 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.303035 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:00.305509 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.305487 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gx2mr\"" Jul 14 15:56:00.311745 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.311723 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfxtb"] Jul 14 15:56:00.393880 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.393843 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf4mg\" (UniqueName: \"kubernetes.io/projected/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-kube-api-access-jf4mg\") pod \"limitador-limitador-64c94fd64f-k2w57\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.394054 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.393917 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-config-file\") pod \"limitador-limitador-64c94fd64f-k2w57\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.394543 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.394524 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-config-file\") pod \"limitador-limitador-64c94fd64f-k2w57\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.401789 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.401762 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf4mg\" (UniqueName: \"kubernetes.io/projected/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-kube-api-access-jf4mg\") pod \"limitador-limitador-64c94fd64f-k2w57\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.494531 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.494439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkrf8\" (UniqueName: \"kubernetes.io/projected/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b-kube-api-access-mkrf8\") pod \"authorino-58cd955675-tfxtb\" (UID: \"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b\") " pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:00.499358 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.499335 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:00.596185 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.595745 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkrf8\" (UniqueName: \"kubernetes.io/projected/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b-kube-api-access-mkrf8\") pod \"authorino-58cd955675-tfxtb\" (UID: \"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b\") " pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:00.605911 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.605870 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkrf8\" (UniqueName: \"kubernetes.io/projected/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b-kube-api-access-mkrf8\") pod \"authorino-58cd955675-tfxtb\" (UID: \"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b\") " pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:00.612528 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.612487 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:00.648941 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.648917 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:00.652331 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:00.652301 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd666f2bb_7ffb_468e_9c02_3dbd9ef36763.slice/crio-e700adfa6e6c7724bc28b07230d44442e888396fc97180da6e2173530a003644 WatchSource:0}: Error finding container e700adfa6e6c7724bc28b07230d44442e888396fc97180da6e2173530a003644: Status 404 returned error can't find the container with id e700adfa6e6c7724bc28b07230d44442e888396fc97180da6e2173530a003644 Jul 14 15:56:00.753906 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.753877 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfxtb"] Jul 14 15:56:00.756550 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:00.756524 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb4cfa10_9e3e_4fa2_9e5b_713f8328ee9b.slice/crio-e58365034e457bfcd0fd90ae4832cb704581d769bb8877a7c32d0c0510e11f80 WatchSource:0}: Error finding container e58365034e457bfcd0fd90ae4832cb704581d769bb8877a7c32d0c0510e11f80: Status 404 returned error can't find the container with id e58365034e457bfcd0fd90ae4832cb704581d769bb8877a7c32d0c0510e11f80 Jul 14 15:56:00.825069 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.825012 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-r46k2"] Jul 14 15:56:00.827824 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.827802 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:00.857628 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.857587 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r46k2"] Jul 14 15:56:00.998905 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:00.998863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl9cf\" (UniqueName: \"kubernetes.io/projected/0c1221e1-342e-43aa-b9ea-ce4d29b21044-kube-api-access-gl9cf\") pod \"authorino-75d556bd77-r46k2\" (UID: \"0c1221e1-342e-43aa-b9ea-ce4d29b21044\") " pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:01.099856 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.099775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl9cf\" (UniqueName: \"kubernetes.io/projected/0c1221e1-342e-43aa-b9ea-ce4d29b21044-kube-api-access-gl9cf\") pod \"authorino-75d556bd77-r46k2\" (UID: \"0c1221e1-342e-43aa-b9ea-ce4d29b21044\") " pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:01.107625 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.107602 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl9cf\" (UniqueName: \"kubernetes.io/projected/0c1221e1-342e-43aa-b9ea-ce4d29b21044-kube-api-access-gl9cf\") pod \"authorino-75d556bd77-r46k2\" (UID: \"0c1221e1-342e-43aa-b9ea-ce4d29b21044\") " pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:01.137449 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.137416 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:01.281535 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.281508 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r46k2"] Jul 14 15:56:01.283800 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:01.283772 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c1221e1_342e_43aa_b9ea_ce4d29b21044.slice/crio-cb095df83549272d9e0f80d538699c8eda2f8061f95cadfb3bcbd3a4d0231c1e WatchSource:0}: Error finding container cb095df83549272d9e0f80d538699c8eda2f8061f95cadfb3bcbd3a4d0231c1e: Status 404 returned error can't find the container with id cb095df83549272d9e0f80d538699c8eda2f8061f95cadfb3bcbd3a4d0231c1e Jul 14 15:56:01.420361 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.420292 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" event={"ID":"d666f2bb-7ffb-468e-9c02-3dbd9ef36763","Type":"ContainerStarted","Data":"e700adfa6e6c7724bc28b07230d44442e888396fc97180da6e2173530a003644"} Jul 14 15:56:01.423276 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.423191 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r46k2" event={"ID":"0c1221e1-342e-43aa-b9ea-ce4d29b21044","Type":"ContainerStarted","Data":"cb095df83549272d9e0f80d538699c8eda2f8061f95cadfb3bcbd3a4d0231c1e"} Jul 14 15:56:01.426759 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:01.426732 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfxtb" event={"ID":"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b","Type":"ContainerStarted","Data":"e58365034e457bfcd0fd90ae4832cb704581d769bb8877a7c32d0c0510e11f80"} Jul 14 15:56:06.461367 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.461329 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r46k2" event={"ID":"0c1221e1-342e-43aa-b9ea-ce4d29b21044","Type":"ContainerStarted","Data":"3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c"} Jul 14 15:56:06.462879 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.462850 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfxtb" event={"ID":"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b","Type":"ContainerStarted","Data":"ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310"} Jul 14 15:56:06.464140 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.464120 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" event={"ID":"d666f2bb-7ffb-468e-9c02-3dbd9ef36763","Type":"ContainerStarted","Data":"3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331"} Jul 14 15:56:06.464268 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.464253 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:06.479058 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.478996 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-r46k2" podStartSLOduration=2.067810077 podStartE2EDuration="6.478981314s" totalImagesPullingTime="4.411171237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:06.478523538 +0000 UTC m=+561.864610925" watchObservedRunningTime="2026-07-14 15:56:06.478981314 +0000 UTC m=+561.865068701" Jul 14 15:56:06.496437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.496376 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" podStartSLOduration=1.415864088 podStartE2EDuration="6.496352706s" totalImagesPullingTime="5.080488618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:06.493717919 +0000 UTC m=+561.879805306" watchObservedRunningTime="2026-07-14 15:56:06.496352706 +0000 UTC m=+561.882440094" Jul 14 15:56:06.508188 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.508156 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfxtb"] Jul 14 15:56:06.517558 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:06.517504 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-tfxtb" podStartSLOduration=1.549277193 podStartE2EDuration="6.51747405s" totalImagesPullingTime="4.968196857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:06.517449978 +0000 UTC m=+561.903537373" watchObservedRunningTime="2026-07-14 15:56:06.51747405 +0000 UTC m=+561.903561439" Jul 14 15:56:08.475677 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:08.475636 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-tfxtb" podUID="cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" containerName="authorino" containerID="cri-o://ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310" gracePeriod=30 Jul 14 15:56:08.737688 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:08.737609 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:08.776339 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:08.776308 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkrf8\" (UniqueName: \"kubernetes.io/projected/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b-kube-api-access-mkrf8\") pod \"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b\" (UID: \"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b\") " Jul 14 15:56:08.778529 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:08.778502 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b-kube-api-access-mkrf8" (OuterVolumeSpecName: "kube-api-access-mkrf8") pod "cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" (UID: "cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b"). InnerVolumeSpecName "kube-api-access-mkrf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:56:08.877201 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:08.877167 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkrf8\" (UniqueName: \"kubernetes.io/projected/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b-kube-api-access-mkrf8\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:09.481552 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.481511 2573 generic.go:356] "Generic (PLEG): container finished" podID="cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" containerID="ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310" exitCode=0 Jul 14 15:56:09.481552 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.481546 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfxtb" Jul 14 15:56:09.482071 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.481587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfxtb" event={"ID":"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b","Type":"ContainerDied","Data":"ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310"} Jul 14 15:56:09.482071 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.481628 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfxtb" event={"ID":"cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b","Type":"ContainerDied","Data":"e58365034e457bfcd0fd90ae4832cb704581d769bb8877a7c32d0c0510e11f80"} Jul 14 15:56:09.482071 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.481648 2573 scope.go:117] "RemoveContainer" containerID="ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310" Jul 14 15:56:09.489694 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.489679 2573 scope.go:117] "RemoveContainer" containerID="ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310" Jul 14 15:56:09.489956 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:56:09.489939 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310\": container with ID starting with ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310 not found: ID does not exist" containerID="ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310" Jul 14 15:56:09.490009 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.489965 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310"} err="failed to get container status \"ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310\": rpc error: code = NotFound desc = could not find container \"ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310\": container with ID starting with ae6d9dccb691ce92fb1599e655710ad1174d596dd2148e1f979d7f8f114b0310 not found: ID does not exist" Jul 14 15:56:09.496605 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.496584 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfxtb"] Jul 14 15:56:09.499795 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:09.499776 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfxtb"] Jul 14 15:56:11.098620 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:11.098589 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" path="/var/lib/kubelet/pods/cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b/volumes" Jul 14 15:56:15.564648 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:15.564616 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:15.565144 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:15.564799 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" podUID="d666f2bb-7ffb-468e-9c02-3dbd9ef36763" containerName="limitador" containerID="cri-o://3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331" gracePeriod=30 Jul 14 15:56:15.565614 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:15.565593 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:16.124543 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.124484 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:16.241057 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.241012 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-config-file\") pod \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " Jul 14 15:56:16.241246 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.241083 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf4mg\" (UniqueName: \"kubernetes.io/projected/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-kube-api-access-jf4mg\") pod \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\" (UID: \"d666f2bb-7ffb-468e-9c02-3dbd9ef36763\") " Jul 14 15:56:16.241394 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.241372 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-config-file" (OuterVolumeSpecName: "config-file") pod "d666f2bb-7ffb-468e-9c02-3dbd9ef36763" (UID: "d666f2bb-7ffb-468e-9c02-3dbd9ef36763"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 15:56:16.243215 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.243188 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-kube-api-access-jf4mg" (OuterVolumeSpecName: "kube-api-access-jf4mg") pod "d666f2bb-7ffb-468e-9c02-3dbd9ef36763" (UID: "d666f2bb-7ffb-468e-9c02-3dbd9ef36763"). InnerVolumeSpecName "kube-api-access-jf4mg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:56:16.342286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.342256 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-config-file\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:16.342286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.342281 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf4mg\" (UniqueName: \"kubernetes.io/projected/d666f2bb-7ffb-468e-9c02-3dbd9ef36763-kube-api-access-jf4mg\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:16.511297 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.511262 2573 generic.go:356] "Generic (PLEG): container finished" podID="d666f2bb-7ffb-468e-9c02-3dbd9ef36763" containerID="3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331" exitCode=0 Jul 14 15:56:16.511476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.511309 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" Jul 14 15:56:16.511476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.511312 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" event={"ID":"d666f2bb-7ffb-468e-9c02-3dbd9ef36763","Type":"ContainerDied","Data":"3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331"} Jul 14 15:56:16.511476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.511434 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-k2w57" event={"ID":"d666f2bb-7ffb-468e-9c02-3dbd9ef36763","Type":"ContainerDied","Data":"e700adfa6e6c7724bc28b07230d44442e888396fc97180da6e2173530a003644"} Jul 14 15:56:16.511476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.511469 2573 scope.go:117] "RemoveContainer" containerID="3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331" Jul 14 15:56:16.520128 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.520109 2573 scope.go:117] "RemoveContainer" containerID="3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331" Jul 14 15:56:16.520408 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:56:16.520391 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331\": container with ID starting with 3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331 not found: ID does not exist" containerID="3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331" Jul 14 15:56:16.520453 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.520420 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331"} err="failed to get container status \"3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331\": rpc error: code = NotFound desc = could not find container \"3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331\": container with ID starting with 3bec630b8026a2a9da24c3275f1a03c2f05bde13c1a5bd99d873a50c7e763331 not found: ID does not exist" Jul 14 15:56:16.532259 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.532225 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:16.533613 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:16.533595 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-k2w57"] Jul 14 15:56:17.098907 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:17.098877 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d666f2bb-7ffb-468e-9c02-3dbd9ef36763" path="/var/lib/kubelet/pods/d666f2bb-7ffb-468e-9c02-3dbd9ef36763/volumes" Jul 14 15:56:21.266528 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266489 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zccxf"] Jul 14 15:56:21.266890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266791 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" containerName="authorino" Jul 14 15:56:21.266890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266801 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" containerName="authorino" Jul 14 15:56:21.266890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266811 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d666f2bb-7ffb-468e-9c02-3dbd9ef36763" containerName="limitador" Jul 14 15:56:21.266890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266816 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d666f2bb-7ffb-468e-9c02-3dbd9ef36763" containerName="limitador" Jul 14 15:56:21.266890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266877 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d666f2bb-7ffb-468e-9c02-3dbd9ef36763" containerName="limitador" Jul 14 15:56:21.266890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.266886 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb4cfa10-9e3e-4fa2-9e5b-713f8328ee9b" containerName="authorino" Jul 14 15:56:21.271130 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.271106 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.273461 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.273441 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 15:56:21.274238 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.274217 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 14 15:56:21.274349 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.274220 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 15:56:21.274349 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.274247 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6zffk\"" Jul 14 15:56:21.278117 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.278099 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zccxf"] Jul 14 15:56:21.382492 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.382459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d1c613f9-54e8-4209-b515-e1d04759a91f-data\") pod \"postgres-69fb4594cf-zccxf\" (UID: \"d1c613f9-54e8-4209-b515-e1d04759a91f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.382666 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.382524 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9sgv\" (UniqueName: \"kubernetes.io/projected/d1c613f9-54e8-4209-b515-e1d04759a91f-kube-api-access-l9sgv\") pod \"postgres-69fb4594cf-zccxf\" (UID: \"d1c613f9-54e8-4209-b515-e1d04759a91f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.483526 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.483489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9sgv\" (UniqueName: \"kubernetes.io/projected/d1c613f9-54e8-4209-b515-e1d04759a91f-kube-api-access-l9sgv\") pod \"postgres-69fb4594cf-zccxf\" (UID: \"d1c613f9-54e8-4209-b515-e1d04759a91f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.483707 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.483545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d1c613f9-54e8-4209-b515-e1d04759a91f-data\") pod \"postgres-69fb4594cf-zccxf\" (UID: \"d1c613f9-54e8-4209-b515-e1d04759a91f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.483883 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.483867 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d1c613f9-54e8-4209-b515-e1d04759a91f-data\") pod \"postgres-69fb4594cf-zccxf\" (UID: \"d1c613f9-54e8-4209-b515-e1d04759a91f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.495918 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.495885 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9sgv\" (UniqueName: \"kubernetes.io/projected/d1c613f9-54e8-4209-b515-e1d04759a91f-kube-api-access-l9sgv\") pod \"postgres-69fb4594cf-zccxf\" (UID: \"d1c613f9-54e8-4209-b515-e1d04759a91f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.582806 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.582715 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:21.717165 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:21.717134 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zccxf"] Jul 14 15:56:21.719694 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:21.719664 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c613f9_54e8_4209_b515_e1d04759a91f.slice/crio-e367dac2169f7f41a6c7add5090cb89185c1ac872435d1021866d736ae49e652 WatchSource:0}: Error finding container e367dac2169f7f41a6c7add5090cb89185c1ac872435d1021866d736ae49e652: Status 404 returned error can't find the container with id e367dac2169f7f41a6c7add5090cb89185c1ac872435d1021866d736ae49e652 Jul 14 15:56:22.542858 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:22.542820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" event={"ID":"d1c613f9-54e8-4209-b515-e1d04759a91f","Type":"ContainerStarted","Data":"e367dac2169f7f41a6c7add5090cb89185c1ac872435d1021866d736ae49e652"} Jul 14 15:56:27.569604 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:27.569562 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" event={"ID":"d1c613f9-54e8-4209-b515-e1d04759a91f","Type":"ContainerStarted","Data":"bbfae49d9d51a60d66ad0eac39e52ddbbb5cda218808e2334921c1cfb4907f25"} Jul 14 15:56:27.569989 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:27.569661 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:27.587713 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:27.587661 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" podStartSLOduration=1.756702879 podStartE2EDuration="6.587645887s" totalImagesPullingTime="4.830943008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:27.586627804 +0000 UTC m=+582.972715212" watchObservedRunningTime="2026-07-14 15:56:27.587645887 +0000 UTC m=+582.973733274" Jul 14 15:56:33.604950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:33.604920 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zccxf" Jul 14 15:56:34.191788 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.191748 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-897fx"] Jul 14 15:56:34.199231 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.199199 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:34.203709 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.203678 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-897fx"] Jul 14 15:56:34.300810 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.300776 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfjtj\" (UniqueName: \"kubernetes.io/projected/9c1397cc-c8ce-4942-b90d-bd95d774fdb8-kube-api-access-bfjtj\") pod \"authorino-5d4b67c446-897fx\" (UID: \"9c1397cc-c8ce-4942-b90d-bd95d774fdb8\") " pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:34.402279 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.402243 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfjtj\" (UniqueName: \"kubernetes.io/projected/9c1397cc-c8ce-4942-b90d-bd95d774fdb8-kube-api-access-bfjtj\") pod \"authorino-5d4b67c446-897fx\" (UID: \"9c1397cc-c8ce-4942-b90d-bd95d774fdb8\") " pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:34.412153 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.412123 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfjtj\" (UniqueName: \"kubernetes.io/projected/9c1397cc-c8ce-4942-b90d-bd95d774fdb8-kube-api-access-bfjtj\") pod \"authorino-5d4b67c446-897fx\" (UID: \"9c1397cc-c8ce-4942-b90d-bd95d774fdb8\") " pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:34.452280 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.452185 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-897fx"] Jul 14 15:56:34.452483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.452466 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:34.480032 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.479998 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78579b757f-z9b8r"] Jul 14 15:56:34.485021 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.485000 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:34.490110 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.490082 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78579b757f-z9b8r"] Jul 14 15:56:34.591877 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.591847 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78579b757f-z9b8r"] Jul 14 15:56:34.592121 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:56:34.592102 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-w2mgx], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-78579b757f-z9b8r" podUID="938f0fe1-24f5-4349-8e93-cad010ea06ca" Jul 14 15:56:34.595180 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.595128 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-897fx"] Jul 14 15:56:34.597307 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:34.597279 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c1397cc_c8ce_4942_b90d_bd95d774fdb8.slice/crio-0c9c20f27cc6e97750726a87223cb43ae02b82ec9840d0a9513f809ea8b2bbf1 WatchSource:0}: Error finding container 0c9c20f27cc6e97750726a87223cb43ae02b82ec9840d0a9513f809ea8b2bbf1: Status 404 returned error can't find the container with id 0c9c20f27cc6e97750726a87223cb43ae02b82ec9840d0a9513f809ea8b2bbf1 Jul 14 15:56:34.603308 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.603278 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2mgx\" (UniqueName: \"kubernetes.io/projected/938f0fe1-24f5-4349-8e93-cad010ea06ca-kube-api-access-w2mgx\") pod \"authorino-78579b757f-z9b8r\" (UID: \"938f0fe1-24f5-4349-8e93-cad010ea06ca\") " pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:34.603625 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.603598 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-897fx" event={"ID":"9c1397cc-c8ce-4942-b90d-bd95d774fdb8","Type":"ContainerStarted","Data":"0c9c20f27cc6e97750726a87223cb43ae02b82ec9840d0a9513f809ea8b2bbf1"} Jul 14 15:56:34.603713 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.603640 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:34.611412 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.611383 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:34.620350 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.620323 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6df6ff8cf9-dbftd"] Jul 14 15:56:34.623553 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.623518 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.626106 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.626085 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 15:56:34.635616 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.635589 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6df6ff8cf9-dbftd"] Jul 14 15:56:34.703861 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.703771 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97775eb3-ce21-457d-b721-af3a602aa5ce-tls-cert\") pod \"authorino-6df6ff8cf9-dbftd\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.703861 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.703820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2mgx\" (UniqueName: \"kubernetes.io/projected/938f0fe1-24f5-4349-8e93-cad010ea06ca-kube-api-access-w2mgx\") pod \"authorino-78579b757f-z9b8r\" (UID: \"938f0fe1-24f5-4349-8e93-cad010ea06ca\") " pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:34.704156 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.703898 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7q7j\" (UniqueName: \"kubernetes.io/projected/97775eb3-ce21-457d-b721-af3a602aa5ce-kube-api-access-v7q7j\") pod \"authorino-6df6ff8cf9-dbftd\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.712879 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.712853 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2mgx\" (UniqueName: \"kubernetes.io/projected/938f0fe1-24f5-4349-8e93-cad010ea06ca-kube-api-access-w2mgx\") pod \"authorino-78579b757f-z9b8r\" (UID: \"938f0fe1-24f5-4349-8e93-cad010ea06ca\") " pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:34.804798 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.804764 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2mgx\" (UniqueName: \"kubernetes.io/projected/938f0fe1-24f5-4349-8e93-cad010ea06ca-kube-api-access-w2mgx\") pod \"938f0fe1-24f5-4349-8e93-cad010ea06ca\" (UID: \"938f0fe1-24f5-4349-8e93-cad010ea06ca\") " Jul 14 15:56:34.804986 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.804906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7q7j\" (UniqueName: \"kubernetes.io/projected/97775eb3-ce21-457d-b721-af3a602aa5ce-kube-api-access-v7q7j\") pod \"authorino-6df6ff8cf9-dbftd\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.804986 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.804950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97775eb3-ce21-457d-b721-af3a602aa5ce-tls-cert\") pod \"authorino-6df6ff8cf9-dbftd\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.807142 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.807109 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/938f0fe1-24f5-4349-8e93-cad010ea06ca-kube-api-access-w2mgx" (OuterVolumeSpecName: "kube-api-access-w2mgx") pod "938f0fe1-24f5-4349-8e93-cad010ea06ca" (UID: "938f0fe1-24f5-4349-8e93-cad010ea06ca"). InnerVolumeSpecName "kube-api-access-w2mgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:56:34.807383 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.807365 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97775eb3-ce21-457d-b721-af3a602aa5ce-tls-cert\") pod \"authorino-6df6ff8cf9-dbftd\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.813031 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.813007 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7q7j\" (UniqueName: \"kubernetes.io/projected/97775eb3-ce21-457d-b721-af3a602aa5ce-kube-api-access-v7q7j\") pod \"authorino-6df6ff8cf9-dbftd\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:34.906376 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.906342 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2mgx\" (UniqueName: \"kubernetes.io/projected/938f0fe1-24f5-4349-8e93-cad010ea06ca-kube-api-access-w2mgx\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:34.932393 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:34.932362 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:56:35.064922 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.064896 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6df6ff8cf9-dbftd"] Jul 14 15:56:35.067879 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:35.067845 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97775eb3_ce21_457d_b721_af3a602aa5ce.slice/crio-776b7beb503e1f5ea57c2900f1748737b6f4fd9b476e8d94059f4a7ce35909cc WatchSource:0}: Error finding container 776b7beb503e1f5ea57c2900f1748737b6f4fd9b476e8d94059f4a7ce35909cc: Status 404 returned error can't find the container with id 776b7beb503e1f5ea57c2900f1748737b6f4fd9b476e8d94059f4a7ce35909cc Jul 14 15:56:35.615201 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.615152 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" event={"ID":"97775eb3-ce21-457d-b721-af3a602aa5ce","Type":"ContainerStarted","Data":"eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1"} Jul 14 15:56:35.615201 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.615202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" event={"ID":"97775eb3-ce21-457d-b721-af3a602aa5ce","Type":"ContainerStarted","Data":"776b7beb503e1f5ea57c2900f1748737b6f4fd9b476e8d94059f4a7ce35909cc"} Jul 14 15:56:35.616927 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.616886 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-897fx" event={"ID":"9c1397cc-c8ce-4942-b90d-bd95d774fdb8","Type":"ContainerStarted","Data":"6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46"} Jul 14 15:56:35.617071 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.616921 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-897fx" podUID="9c1397cc-c8ce-4942-b90d-bd95d774fdb8" containerName="authorino" containerID="cri-o://6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46" gracePeriod=30 Jul 14 15:56:35.617071 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.616937 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78579b757f-z9b8r" Jul 14 15:56:35.631103 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.631037 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" podStartSLOduration=1.306872439 podStartE2EDuration="1.631018549s" totalImagesPullingTime="324.14611ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:35.630088761 +0000 UTC m=+591.016176170" watchObservedRunningTime="2026-07-14 15:56:35.631018549 +0000 UTC m=+591.017105937" Jul 14 15:56:35.654005 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.653928 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78579b757f-z9b8r"] Jul 14 15:56:35.661380 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.661347 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r46k2"] Jul 14 15:56:35.661545 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.661523 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-r46k2" podUID="0c1221e1-342e-43aa-b9ea-ce4d29b21044" containerName="authorino" containerID="cri-o://3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c" gracePeriod=30 Jul 14 15:56:35.663327 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.663306 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78579b757f-z9b8r"] Jul 14 15:56:35.675924 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.675846 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-897fx" podStartSLOduration=1.279279575 podStartE2EDuration="1.675824645s" totalImagesPullingTime="396.54507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:35.674317928 +0000 UTC m=+591.060405316" watchObservedRunningTime="2026-07-14 15:56:35.675824645 +0000 UTC m=+591.061912030" Jul 14 15:56:35.923267 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.923242 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:35.926326 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:35.926305 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:36.015783 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.015748 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfjtj\" (UniqueName: \"kubernetes.io/projected/9c1397cc-c8ce-4942-b90d-bd95d774fdb8-kube-api-access-bfjtj\") pod \"9c1397cc-c8ce-4942-b90d-bd95d774fdb8\" (UID: \"9c1397cc-c8ce-4942-b90d-bd95d774fdb8\") " Jul 14 15:56:36.015943 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.015816 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl9cf\" (UniqueName: \"kubernetes.io/projected/0c1221e1-342e-43aa-b9ea-ce4d29b21044-kube-api-access-gl9cf\") pod \"0c1221e1-342e-43aa-b9ea-ce4d29b21044\" (UID: \"0c1221e1-342e-43aa-b9ea-ce4d29b21044\") " Jul 14 15:56:36.017977 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.017945 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c1397cc-c8ce-4942-b90d-bd95d774fdb8-kube-api-access-bfjtj" (OuterVolumeSpecName: "kube-api-access-bfjtj") pod "9c1397cc-c8ce-4942-b90d-bd95d774fdb8" (UID: "9c1397cc-c8ce-4942-b90d-bd95d774fdb8"). InnerVolumeSpecName "kube-api-access-bfjtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:56:36.018112 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.017945 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1221e1-342e-43aa-b9ea-ce4d29b21044-kube-api-access-gl9cf" (OuterVolumeSpecName: "kube-api-access-gl9cf") pod "0c1221e1-342e-43aa-b9ea-ce4d29b21044" (UID: "0c1221e1-342e-43aa-b9ea-ce4d29b21044"). InnerVolumeSpecName "kube-api-access-gl9cf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:56:36.116385 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.116345 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfjtj\" (UniqueName: \"kubernetes.io/projected/9c1397cc-c8ce-4942-b90d-bd95d774fdb8-kube-api-access-bfjtj\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:36.116385 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.116378 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl9cf\" (UniqueName: \"kubernetes.io/projected/0c1221e1-342e-43aa-b9ea-ce4d29b21044-kube-api-access-gl9cf\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:36.622272 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.622230 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c1221e1-342e-43aa-b9ea-ce4d29b21044" containerID="3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c" exitCode=0 Jul 14 15:56:36.622272 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.622273 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-r46k2" Jul 14 15:56:36.622788 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.622318 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r46k2" event={"ID":"0c1221e1-342e-43aa-b9ea-ce4d29b21044","Type":"ContainerDied","Data":"3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c"} Jul 14 15:56:36.622788 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.622371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-r46k2" event={"ID":"0c1221e1-342e-43aa-b9ea-ce4d29b21044","Type":"ContainerDied","Data":"cb095df83549272d9e0f80d538699c8eda2f8061f95cadfb3bcbd3a4d0231c1e"} Jul 14 15:56:36.622788 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.622396 2573 scope.go:117] "RemoveContainer" containerID="3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c" Jul 14 15:56:36.624183 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.623828 2573 generic.go:356] "Generic (PLEG): container finished" podID="9c1397cc-c8ce-4942-b90d-bd95d774fdb8" containerID="6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46" exitCode=0 Jul 14 15:56:36.624183 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.623881 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-897fx" Jul 14 15:56:36.624183 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.623917 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-897fx" event={"ID":"9c1397cc-c8ce-4942-b90d-bd95d774fdb8","Type":"ContainerDied","Data":"6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46"} Jul 14 15:56:36.624183 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.623954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-897fx" event={"ID":"9c1397cc-c8ce-4942-b90d-bd95d774fdb8","Type":"ContainerDied","Data":"0c9c20f27cc6e97750726a87223cb43ae02b82ec9840d0a9513f809ea8b2bbf1"} Jul 14 15:56:36.632251 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.632233 2573 scope.go:117] "RemoveContainer" containerID="3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c" Jul 14 15:56:36.632519 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:56:36.632500 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c\": container with ID starting with 3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c not found: ID does not exist" containerID="3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c" Jul 14 15:56:36.632609 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.632532 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c"} err="failed to get container status \"3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c\": rpc error: code = NotFound desc = could not find container \"3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c\": container with ID starting with 3e81b44710c7efad1fd48660df2ff4f8c7323d24d83ddc4fc638741a76cb0b5c not found: ID does not exist" Jul 14 15:56:36.632609 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.632555 2573 scope.go:117] "RemoveContainer" containerID="6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46" Jul 14 15:56:36.643171 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.643152 2573 scope.go:117] "RemoveContainer" containerID="6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46" Jul 14 15:56:36.643474 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:56:36.643447 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46\": container with ID starting with 6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46 not found: ID does not exist" containerID="6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46" Jul 14 15:56:36.643474 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.643474 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46"} err="failed to get container status \"6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46\": rpc error: code = NotFound desc = could not find container \"6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46\": container with ID starting with 6b66742d6dc892fc219c22df28dbb1ed2b910b17f4602ab1ec69e43d89faca46 not found: ID does not exist" Jul 14 15:56:36.645303 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.645277 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r46k2"] Jul 14 15:56:36.649193 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.649172 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-r46k2"] Jul 14 15:56:36.658562 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.658533 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-897fx"] Jul 14 15:56:36.660401 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:36.660381 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-897fx"] Jul 14 15:56:37.098977 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:37.098943 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c1221e1-342e-43aa-b9ea-ce4d29b21044" path="/var/lib/kubelet/pods/0c1221e1-342e-43aa-b9ea-ce4d29b21044/volumes" Jul 14 15:56:37.099435 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:37.099415 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="938f0fe1-24f5-4349-8e93-cad010ea06ca" path="/var/lib/kubelet/pods/938f0fe1-24f5-4349-8e93-cad010ea06ca/volumes" Jul 14 15:56:37.099695 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:37.099680 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c1397cc-c8ce-4942-b90d-bd95d774fdb8" path="/var/lib/kubelet/pods/9c1397cc-c8ce-4942-b90d-bd95d774fdb8/volumes" Jul 14 15:56:40.420563 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.420531 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b58f466b6-9p5pf"] Jul 14 15:56:40.421100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.421029 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c1397cc-c8ce-4942-b90d-bd95d774fdb8" containerName="authorino" Jul 14 15:56:40.421100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.421069 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c1397cc-c8ce-4942-b90d-bd95d774fdb8" containerName="authorino" Jul 14 15:56:40.421100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.421089 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1221e1-342e-43aa-b9ea-ce4d29b21044" containerName="authorino" Jul 14 15:56:40.421100 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.421098 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1221e1-342e-43aa-b9ea-ce4d29b21044" containerName="authorino" Jul 14 15:56:40.421241 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.421211 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c1397cc-c8ce-4942-b90d-bd95d774fdb8" containerName="authorino" Jul 14 15:56:40.421241 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.421234 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1221e1-342e-43aa-b9ea-ce4d29b21044" containerName="authorino" Jul 14 15:56:40.464174 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.464134 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b58f466b6-9p5pf"] Jul 14 15:56:40.464366 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.464281 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.466607 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.466576 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 15:56:40.466769 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.466587 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 15:56:40.466769 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.466647 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-279d7\"" Jul 14 15:56:40.553871 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.553832 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-649vh\" (UniqueName: \"kubernetes.io/projected/ce03e729-1ee8-4762-8c28-b208732a1f37-kube-api-access-649vh\") pod \"maas-controller-b58f466b6-9p5pf\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.554098 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.553969 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce03e729-1ee8-4762-8c28-b208732a1f37-webhook-cert\") pod \"maas-controller-b58f466b6-9p5pf\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.599531 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.599496 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c4db64696-q2kpl"] Jul 14 15:56:40.618090 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.618034 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.630492 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.630464 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c4db64696-q2kpl"] Jul 14 15:56:40.654511 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.654478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce03e729-1ee8-4762-8c28-b208732a1f37-webhook-cert\") pod \"maas-controller-b58f466b6-9p5pf\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.654679 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.654520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-649vh\" (UniqueName: \"kubernetes.io/projected/ce03e729-1ee8-4762-8c28-b208732a1f37-kube-api-access-649vh\") pod \"maas-controller-b58f466b6-9p5pf\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.657033 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.657010 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce03e729-1ee8-4762-8c28-b208732a1f37-webhook-cert\") pod \"maas-controller-b58f466b6-9p5pf\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.663877 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.663844 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-649vh\" (UniqueName: \"kubernetes.io/projected/ce03e729-1ee8-4762-8c28-b208732a1f37-kube-api-access-649vh\") pod \"maas-controller-b58f466b6-9p5pf\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.755691 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.755592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0eec778-0478-4a1f-9203-edc6f41d1eed-webhook-cert\") pod \"maas-controller-7c4db64696-q2kpl\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.755691 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.755642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp24s\" (UniqueName: \"kubernetes.io/projected/a0eec778-0478-4a1f-9203-edc6f41d1eed-kube-api-access-tp24s\") pod \"maas-controller-7c4db64696-q2kpl\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.777681 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.777644 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:40.856851 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.856816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0eec778-0478-4a1f-9203-edc6f41d1eed-webhook-cert\") pod \"maas-controller-7c4db64696-q2kpl\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.857030 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.856884 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp24s\" (UniqueName: \"kubernetes.io/projected/a0eec778-0478-4a1f-9203-edc6f41d1eed-kube-api-access-tp24s\") pod \"maas-controller-7c4db64696-q2kpl\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.859703 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.859601 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0eec778-0478-4a1f-9203-edc6f41d1eed-webhook-cert\") pod \"maas-controller-7c4db64696-q2kpl\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.866791 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.866763 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp24s\" (UniqueName: \"kubernetes.io/projected/a0eec778-0478-4a1f-9203-edc6f41d1eed-kube-api-access-tp24s\") pod \"maas-controller-7c4db64696-q2kpl\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:40.913055 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.913007 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b58f466b6-9p5pf"] Jul 14 15:56:40.915729 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:40.915699 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce03e729_1ee8_4762_8c28_b208732a1f37.slice/crio-08410428e444b26ecdff552bc39ad0055cb8986850ba226d76abf34a0b3cc9f7 WatchSource:0}: Error finding container 08410428e444b26ecdff552bc39ad0055cb8986850ba226d76abf34a0b3cc9f7: Status 404 returned error can't find the container with id 08410428e444b26ecdff552bc39ad0055cb8986850ba226d76abf34a0b3cc9f7 Jul 14 15:56:40.928439 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:40.928416 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:41.062126 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:41.062102 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c4db64696-q2kpl"] Jul 14 15:56:41.064476 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:56:41.064446 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0eec778_0478_4a1f_9203_edc6f41d1eed.slice/crio-7e2fc6e12d4c1c1451bba598e489a8da9a2c82761530387793371570559ddd0b WatchSource:0}: Error finding container 7e2fc6e12d4c1c1451bba598e489a8da9a2c82761530387793371570559ddd0b: Status 404 returned error can't find the container with id 7e2fc6e12d4c1c1451bba598e489a8da9a2c82761530387793371570559ddd0b Jul 14 15:56:41.650664 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:41.650627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c4db64696-q2kpl" event={"ID":"a0eec778-0478-4a1f-9203-edc6f41d1eed","Type":"ContainerStarted","Data":"7e2fc6e12d4c1c1451bba598e489a8da9a2c82761530387793371570559ddd0b"} Jul 14 15:56:41.652058 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:41.652012 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f466b6-9p5pf" event={"ID":"ce03e729-1ee8-4762-8c28-b208732a1f37","Type":"ContainerStarted","Data":"08410428e444b26ecdff552bc39ad0055cb8986850ba226d76abf34a0b3cc9f7"} Jul 14 15:56:43.664103 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:43.664067 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c4db64696-q2kpl" event={"ID":"a0eec778-0478-4a1f-9203-edc6f41d1eed","Type":"ContainerStarted","Data":"ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e"} Jul 14 15:56:43.664594 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:43.664240 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:43.666188 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:43.666158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f466b6-9p5pf" event={"ID":"ce03e729-1ee8-4762-8c28-b208732a1f37","Type":"ContainerStarted","Data":"f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea"} Jul 14 15:56:43.666317 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:43.666253 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:43.684161 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:43.684111 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c4db64696-q2kpl" podStartSLOduration=1.404273444 podStartE2EDuration="3.684090423s" totalImagesPullingTime="2.279816979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:43.682872477 +0000 UTC m=+599.068959886" watchObservedRunningTime="2026-07-14 15:56:43.684090423 +0000 UTC m=+599.070177811" Jul 14 15:56:43.705450 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:43.705381 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b58f466b6-9p5pf" podStartSLOduration=1.273548042 podStartE2EDuration="3.705365798s" totalImagesPullingTime="2.431817756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:56:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:56:43.703698536 +0000 UTC m=+599.089785935" watchObservedRunningTime="2026-07-14 15:56:43.705365798 +0000 UTC m=+599.091453202" Jul 14 15:56:45.029453 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:45.029427 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:56:45.029825 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:45.029426 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 15:56:54.673151 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:54.673120 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:54.673611 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:54.673226 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 15:56:54.745016 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:54.744977 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b58f466b6-9p5pf"] Jul 14 15:56:54.745253 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:54.745228 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b58f466b6-9p5pf" podUID="ce03e729-1ee8-4762-8c28-b208732a1f37" containerName="manager" containerID="cri-o://f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea" gracePeriod=30 Jul 14 15:56:54.994466 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:54.994443 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:55.069177 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.069142 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-649vh\" (UniqueName: \"kubernetes.io/projected/ce03e729-1ee8-4762-8c28-b208732a1f37-kube-api-access-649vh\") pod \"ce03e729-1ee8-4762-8c28-b208732a1f37\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " Jul 14 15:56:55.069362 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.069242 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce03e729-1ee8-4762-8c28-b208732a1f37-webhook-cert\") pod \"ce03e729-1ee8-4762-8c28-b208732a1f37\" (UID: \"ce03e729-1ee8-4762-8c28-b208732a1f37\") " Jul 14 15:56:55.071556 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.071526 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce03e729-1ee8-4762-8c28-b208732a1f37-kube-api-access-649vh" (OuterVolumeSpecName: "kube-api-access-649vh") pod "ce03e729-1ee8-4762-8c28-b208732a1f37" (UID: "ce03e729-1ee8-4762-8c28-b208732a1f37"). InnerVolumeSpecName "kube-api-access-649vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:56:55.071612 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.071536 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce03e729-1ee8-4762-8c28-b208732a1f37-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ce03e729-1ee8-4762-8c28-b208732a1f37" (UID: "ce03e729-1ee8-4762-8c28-b208732a1f37"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:56:55.170521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.170492 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce03e729-1ee8-4762-8c28-b208732a1f37-webhook-cert\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:55.170521 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.170521 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-649vh\" (UniqueName: \"kubernetes.io/projected/ce03e729-1ee8-4762-8c28-b208732a1f37-kube-api-access-649vh\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:56:55.727951 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.727911 2573 generic.go:356] "Generic (PLEG): container finished" podID="ce03e729-1ee8-4762-8c28-b208732a1f37" containerID="f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea" exitCode=0 Jul 14 15:56:55.728432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.727959 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f466b6-9p5pf" Jul 14 15:56:55.728432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.728007 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f466b6-9p5pf" event={"ID":"ce03e729-1ee8-4762-8c28-b208732a1f37","Type":"ContainerDied","Data":"f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea"} Jul 14 15:56:55.728432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.728096 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f466b6-9p5pf" event={"ID":"ce03e729-1ee8-4762-8c28-b208732a1f37","Type":"ContainerDied","Data":"08410428e444b26ecdff552bc39ad0055cb8986850ba226d76abf34a0b3cc9f7"} Jul 14 15:56:55.728432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.728132 2573 scope.go:117] "RemoveContainer" containerID="f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea" Jul 14 15:56:55.736531 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.736513 2573 scope.go:117] "RemoveContainer" containerID="f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea" Jul 14 15:56:55.736861 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:56:55.736840 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea\": container with ID starting with f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea not found: ID does not exist" containerID="f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea" Jul 14 15:56:55.736900 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.736871 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea"} err="failed to get container status \"f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea\": rpc error: code = NotFound desc = could not find container \"f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea\": container with ID starting with f07c2f119a77d71ea6847fff92da1e402cc28fbf563e63eed82ae18a4e4734ea not found: ID does not exist" Jul 14 15:56:55.745450 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.745424 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b58f466b6-9p5pf"] Jul 14 15:56:55.749873 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:55.749849 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b58f466b6-9p5pf"] Jul 14 15:56:57.097989 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:56:57.097946 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce03e729-1ee8-4762-8c28-b208732a1f37" path="/var/lib/kubelet/pods/ce03e729-1ee8-4762-8c28-b208732a1f37/volumes" Jul 14 15:57:16.785134 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.785035 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7"] Jul 14 15:57:16.785584 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.785437 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce03e729-1ee8-4762-8c28-b208732a1f37" containerName="manager" Jul 14 15:57:16.785584 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.785448 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce03e729-1ee8-4762-8c28-b208732a1f37" containerName="manager" Jul 14 15:57:16.785584 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.785513 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce03e729-1ee8-4762-8c28-b208732a1f37" containerName="manager" Jul 14 15:57:16.791494 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.791471 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.793899 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.793873 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 14 15:57:16.800762 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.800711 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7"] Jul 14 15:57:16.851374 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.851341 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36f8b539-eb72-4d7c-a407-434c3323c007-maas-api-tls\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.851535 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.851389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b59ws\" (UniqueName: \"kubernetes.io/projected/36f8b539-eb72-4d7c-a407-434c3323c007-kube-api-access-b59ws\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.851535 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.851422 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36f8b539-eb72-4d7c-a407-434c3323c007-openshift-service-ca\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.952319 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.952287 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36f8b539-eb72-4d7c-a407-434c3323c007-maas-api-tls\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.952487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.952335 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b59ws\" (UniqueName: \"kubernetes.io/projected/36f8b539-eb72-4d7c-a407-434c3323c007-kube-api-access-b59ws\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.952487 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.952457 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36f8b539-eb72-4d7c-a407-434c3323c007-openshift-service-ca\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.953131 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.953108 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36f8b539-eb72-4d7c-a407-434c3323c007-openshift-service-ca\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.955031 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.955006 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36f8b539-eb72-4d7c-a407-434c3323c007-maas-api-tls\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:16.961180 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:16.961156 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b59ws\" (UniqueName: \"kubernetes.io/projected/36f8b539-eb72-4d7c-a407-434c3323c007-kube-api-access-b59ws\") pod \"maas-api-6d7f74d569-6npt7\" (UID: \"36f8b539-eb72-4d7c-a407-434c3323c007\") " pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:17.103290 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.103217 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:17.254920 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.254841 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7"] Jul 14 15:57:17.257389 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:57:17.257359 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f8b539_eb72_4d7c_a407_434c3323c007.slice/crio-b83715e5f62a7e5f59e8e7cc874f2e8b2b2e4a9deaf9bf9ba1af5ce5ea299aff WatchSource:0}: Error finding container b83715e5f62a7e5f59e8e7cc874f2e8b2b2e4a9deaf9bf9ba1af5ce5ea299aff: Status 404 returned error can't find the container with id b83715e5f62a7e5f59e8e7cc874f2e8b2b2e4a9deaf9bf9ba1af5ce5ea299aff Jul 14 15:57:17.442747 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.442711 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-zf6mx"] Jul 14 15:57:17.448358 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.448332 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.450518 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.450492 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 15:57:17.450719 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.450702 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zptcs\"" Jul 14 15:57:17.456714 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.456683 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-zf6mx"] Jul 14 15:57:17.557772 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.557742 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dfdb8d-030b-4220-86d7-16e3126569ea-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-zf6mx\" (UID: \"c4dfdb8d-030b-4220-86d7-16e3126569ea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.557939 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.557801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6rwq\" (UniqueName: \"kubernetes.io/projected/c4dfdb8d-030b-4220-86d7-16e3126569ea-kube-api-access-n6rwq\") pod \"payload-processing-f9b8d9b9b-zf6mx\" (UID: \"c4dfdb8d-030b-4220-86d7-16e3126569ea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.658295 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.658258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6rwq\" (UniqueName: \"kubernetes.io/projected/c4dfdb8d-030b-4220-86d7-16e3126569ea-kube-api-access-n6rwq\") pod \"payload-processing-f9b8d9b9b-zf6mx\" (UID: \"c4dfdb8d-030b-4220-86d7-16e3126569ea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.658483 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.658342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dfdb8d-030b-4220-86d7-16e3126569ea-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-zf6mx\" (UID: \"c4dfdb8d-030b-4220-86d7-16e3126569ea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.659029 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.659008 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dfdb8d-030b-4220-86d7-16e3126569ea-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-zf6mx\" (UID: \"c4dfdb8d-030b-4220-86d7-16e3126569ea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.666502 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.666475 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6rwq\" (UniqueName: \"kubernetes.io/projected/c4dfdb8d-030b-4220-86d7-16e3126569ea-kube-api-access-n6rwq\") pod \"payload-processing-f9b8d9b9b-zf6mx\" (UID: \"c4dfdb8d-030b-4220-86d7-16e3126569ea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.761124 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.761032 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:17.830367 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.830326 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" event={"ID":"36f8b539-eb72-4d7c-a407-434c3323c007","Type":"ContainerStarted","Data":"b83715e5f62a7e5f59e8e7cc874f2e8b2b2e4a9deaf9bf9ba1af5ce5ea299aff"} Jul 14 15:57:17.901651 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:17.901617 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-zf6mx"] Jul 14 15:57:17.904375 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:57:17.904352 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4dfdb8d_030b_4220_86d7_16e3126569ea.slice/crio-37d40d6954b01cbcabc0a70a3ad2aea534ff69940deaddf6327a18e1b7807daa WatchSource:0}: Error finding container 37d40d6954b01cbcabc0a70a3ad2aea534ff69940deaddf6327a18e1b7807daa: Status 404 returned error can't find the container with id 37d40d6954b01cbcabc0a70a3ad2aea534ff69940deaddf6327a18e1b7807daa Jul 14 15:57:18.863560 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:18.863346 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" event={"ID":"c4dfdb8d-030b-4220-86d7-16e3126569ea","Type":"ContainerStarted","Data":"37d40d6954b01cbcabc0a70a3ad2aea534ff69940deaddf6327a18e1b7807daa"} Jul 14 15:57:20.874979 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:20.874929 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" event={"ID":"c4dfdb8d-030b-4220-86d7-16e3126569ea","Type":"ContainerStarted","Data":"085e0487a0d8348f9421f2420e6862689540627f3a08fade18d6fe67b98eb114"} Jul 14 15:57:20.875389 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:20.875157 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:57:20.877194 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:20.877169 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" event={"ID":"36f8b539-eb72-4d7c-a407-434c3323c007","Type":"ContainerStarted","Data":"c2a2318158c43306cddb311fa894a630f1062a0f055114a933b43ede0507070b"} Jul 14 15:57:20.877328 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:20.877266 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:20.894951 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:20.894895 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" podStartSLOduration=1.830992961 podStartE2EDuration="3.894879951s" totalImagesPullingTime="2.06388699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:57:20.893023818 +0000 UTC m=+636.279111207" watchObservedRunningTime="2026-07-14 15:57:20.894879951 +0000 UTC m=+636.280967338" Jul 14 15:57:20.909221 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:20.909166 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" podStartSLOduration=2.20310478 podStartE2EDuration="4.909151331s" totalImagesPullingTime="2.706046551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:57:20.908642941 +0000 UTC m=+636.294730319" watchObservedRunningTime="2026-07-14 15:57:20.909151331 +0000 UTC m=+636.295238719" Jul 14 15:57:25.902688 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:25.902651 2573 generic.go:356] "Generic (PLEG): container finished" podID="8864abe0-921f-41b4-91da-ac151bf2da49" containerID="0bbd73f9124dc0bd7922d1c1e964c2a7b084fb3deb80683c7e30e748d4fa556c" exitCode=0 Jul 14 15:57:25.903126 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:25.902724 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" event={"ID":"8864abe0-921f-41b4-91da-ac151bf2da49","Type":"ContainerDied","Data":"0bbd73f9124dc0bd7922d1c1e964c2a7b084fb3deb80683c7e30e748d4fa556c"} Jul 14 15:57:25.903177 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:25.903135 2573 scope.go:117] "RemoveContainer" containerID="0bbd73f9124dc0bd7922d1c1e964c2a7b084fb3deb80683c7e30e748d4fa556c" Jul 14 15:57:25.904183 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:25.904155 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:57:25.904248 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:25.904233 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:57:25.904284 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:25.904271 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 14 15:57:26.888467 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:26.888433 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d7f74d569-6npt7" Jul 14 15:57:26.909633 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:26.909586 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" event={"ID":"8864abe0-921f-41b4-91da-ac151bf2da49","Type":"ContainerStarted","Data":"071637ebc1301c8e9426048ac9992de17ec1d990d458668895511aac284bcc48"} Jul 14 15:57:27.044424 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:27.044314 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:57:27.046176 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:27.046147 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m2zw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 14 15:57:27.046414 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:27.046374 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" podUID="8864abe0-921f-41b4-91da-ac151bf2da49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 14 15:57:28.044808 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:28.044771 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m2zw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 14 15:57:28.045811 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:28.045408 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" podUID="8864abe0-921f-41b4-91da-ac151bf2da49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 14 15:57:29.044551 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:29.044512 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m2zw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 14 15:57:29.044827 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:29.044798 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" podUID="8864abe0-921f-41b4-91da-ac151bf2da49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 14 15:57:29.405063 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:29.405002 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:57:30.044942 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:30.044903 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m2zw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 14 15:57:30.046003 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:30.045562 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" podUID="8864abe0-921f-41b4-91da-ac151bf2da49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 14 15:57:31.048166 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:31.048133 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:57:31.049147 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:31.049125 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m2zw2" Jul 14 15:57:31.885497 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:57:31.885464 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-zf6mx" Jul 14 15:58:08.300865 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.300825 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms"] Jul 14 15:58:08.304208 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.304192 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.307503 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.307480 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 15:58:08.307619 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.307504 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z2zwk\"" Jul 14 15:58:08.307619 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.307516 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 15:58:08.307619 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.307585 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 15:58:08.314499 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.314474 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms"] Jul 14 15:58:08.393615 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.393569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.393615 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.393615 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.393844 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.393645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.393844 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.393664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5btr9\" (UniqueName: \"kubernetes.io/projected/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-kube-api-access-5btr9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.393844 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.393713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.393844 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.393750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495007 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.494967 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495007 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5btr9\" (UniqueName: \"kubernetes.io/projected/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-kube-api-access-5btr9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495558 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495517 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495558 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495538 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.495710 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.495562 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.497391 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.497364 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.497642 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.497627 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.503367 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.503348 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5btr9\" (UniqueName: \"kubernetes.io/projected/286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6-kube-api-access-5btr9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9wpms\" (UID: \"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.615327 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.615242 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:08.759825 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.759796 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms"] Jul 14 15:58:08.762301 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:58:08.762272 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286ff7c4_3fa3_4248_b61b_f8fcdcf2dfe6.slice/crio-2be59d87519b786693d515bd65eaae9131affc8878b5ba7d6317e05fe6aed5f6 WatchSource:0}: Error finding container 2be59d87519b786693d515bd65eaae9131affc8878b5ba7d6317e05fe6aed5f6: Status 404 returned error can't find the container with id 2be59d87519b786693d515bd65eaae9131affc8878b5ba7d6317e05fe6aed5f6 Jul 14 15:58:08.763615 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:08.763590 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 15:58:09.101337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:09.101263 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" event={"ID":"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6","Type":"ContainerStarted","Data":"2be59d87519b786693d515bd65eaae9131affc8878b5ba7d6317e05fe6aed5f6"} Jul 14 15:58:10.107385 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:10.107350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" event={"ID":"286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6","Type":"ContainerStarted","Data":"01b7bef44791bd352c02d1a4475a03ea95fd6e366f75bad422d8e321546c442a"} Jul 14 15:58:10.126514 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:10.126459 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" podStartSLOduration=0.928115405 podStartE2EDuration="2.126443838s" totalImagesPullingTime="1.198328433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:58:10.125301065 +0000 UTC m=+685.511388451" watchObservedRunningTime="2026-07-14 15:58:10.126443838 +0000 UTC m=+685.512531224" Jul 14 15:58:16.994680 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:16.994648 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7"] Jul 14 15:58:16.998440 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:16.998418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.001077 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.001034 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 15:58:17.008523 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.008500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7"] Jul 14 15:58:17.079145 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.079107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.079312 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.079150 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kf85\" (UniqueName: \"kubernetes.io/projected/504f55b6-d22d-45e2-a8ea-ecc664881ece-kube-api-access-8kf85\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.079312 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.079188 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.079312 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.079227 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.079312 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.079248 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.079452 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.079323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504f55b6-d22d-45e2-a8ea-ecc664881ece-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.179890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.179846 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.179890 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.179895 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kf85\" (UniqueName: \"kubernetes.io/projected/504f55b6-d22d-45e2-a8ea-ecc664881ece-kube-api-access-8kf85\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180185 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.179931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180185 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.179973 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180185 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.180001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180185 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.180075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504f55b6-d22d-45e2-a8ea-ecc664881ece-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180440 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.180408 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180581 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.180497 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.180581 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.180549 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.182554 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.182533 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504f55b6-d22d-45e2-a8ea-ecc664881ece-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.182641 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.182625 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504f55b6-d22d-45e2-a8ea-ecc664881ece-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.194572 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.194545 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kf85\" (UniqueName: \"kubernetes.io/projected/504f55b6-d22d-45e2-a8ea-ecc664881ece-kube-api-access-8kf85\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vjjd7\" (UID: \"504f55b6-d22d-45e2-a8ea-ecc664881ece\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.308885 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.308781 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:17.456003 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:17.455978 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7"] Jul 14 15:58:17.458439 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:58:17.458410 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504f55b6_d22d_45e2_a8ea_ecc664881ece.slice/crio-8ab05a7ec30b4ab10147b55c733f8d66c0ea772c57dfe73b4bfe84f2a840ff5c WatchSource:0}: Error finding container 8ab05a7ec30b4ab10147b55c733f8d66c0ea772c57dfe73b4bfe84f2a840ff5c: Status 404 returned error can't find the container with id 8ab05a7ec30b4ab10147b55c733f8d66c0ea772c57dfe73b4bfe84f2a840ff5c Jul 14 15:58:18.146998 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:18.146962 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" event={"ID":"504f55b6-d22d-45e2-a8ea-ecc664881ece","Type":"ContainerStarted","Data":"3e425a1d261f0929a5c7f6213b0db892678ce52e6418415154a985821b6408f7"} Jul 14 15:58:18.147383 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:18.147019 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" event={"ID":"504f55b6-d22d-45e2-a8ea-ecc664881ece","Type":"ContainerStarted","Data":"8ab05a7ec30b4ab10147b55c733f8d66c0ea772c57dfe73b4bfe84f2a840ff5c"} Jul 14 15:58:18.171950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:18.171898 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" podStartSLOduration=1.9745896109999999 podStartE2EDuration="2.171882459s" totalImagesPullingTime="197.292848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:58:18.171767677 +0000 UTC m=+693.557855066" watchObservedRunningTime="2026-07-14 15:58:18.171882459 +0000 UTC m=+693.557969846" Jul 14 15:58:18.616257 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:18.616148 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:18.617032 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:18.617008 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:18.635722 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:18.635691 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:19.165243 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:19.165204 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9wpms" Jul 14 15:58:22.186337 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.186293 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h"] Jul 14 15:58:22.190529 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.190502 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.193485 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.193466 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 15:58:22.200182 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.200157 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h"] Jul 14 15:58:22.324857 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.324824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.325066 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.324882 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.325066 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.324912 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.325066 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.324972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.325066 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.325006 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf2qk\" (UniqueName: \"kubernetes.io/projected/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-kube-api-access-gf2qk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.325066 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.325024 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426437 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426392 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426610 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426610 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf2qk\" (UniqueName: \"kubernetes.io/projected/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-kube-api-access-gf2qk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426687 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426687 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426663 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426778 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426727 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.426998 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.426977 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.427107 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.427066 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.427168 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.427118 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.428881 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.428861 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.429069 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.429020 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.442854 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.442785 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf2qk\" (UniqueName: \"kubernetes.io/projected/633ba653-b776-4c3f-a4f3-1dfd2a0725a5-kube-api-access-gf2qk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h\" (UID: \"633ba653-b776-4c3f-a4f3-1dfd2a0725a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.502208 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.502167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:22.644610 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:22.642207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h"] Jul 14 15:58:22.645399 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:58:22.645366 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633ba653_b776_4c3f_a4f3_1dfd2a0725a5.slice/crio-028a566b0faa6ab0de2ac24fec69a4a97181e50ce8bd654a5dcf8dc564c3130c WatchSource:0}: Error finding container 028a566b0faa6ab0de2ac24fec69a4a97181e50ce8bd654a5dcf8dc564c3130c: Status 404 returned error can't find the container with id 028a566b0faa6ab0de2ac24fec69a4a97181e50ce8bd654a5dcf8dc564c3130c Jul 14 15:58:23.172697 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:23.172660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" event={"ID":"633ba653-b776-4c3f-a4f3-1dfd2a0725a5","Type":"ContainerStarted","Data":"c9fac7bdabeb160679a8e74e6026b6bb28702fd4737f98040043f2181f3efff6"} Jul 14 15:58:23.172886 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:23.172717 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" event={"ID":"633ba653-b776-4c3f-a4f3-1dfd2a0725a5","Type":"ContainerStarted","Data":"028a566b0faa6ab0de2ac24fec69a4a97181e50ce8bd654a5dcf8dc564c3130c"} Jul 14 15:58:23.191946 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:23.191887 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" podStartSLOduration=1.00445628 podStartE2EDuration="1.191866439s" totalImagesPullingTime="187.410159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:58:23.189927779 +0000 UTC m=+698.576015201" watchObservedRunningTime="2026-07-14 15:58:23.191866439 +0000 UTC m=+698.577953825" Jul 14 15:58:27.309110 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:27.309066 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:27.309507 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:27.309134 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:27.322265 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:27.322237 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:27.994289 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:27.994251 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x"] Jul 14 15:58:27.997952 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:27.997928 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.000581 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.000559 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 15:58:28.010634 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.010610 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x"] Jul 14 15:58:28.079118 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.079074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.079315 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.079125 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.079315 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.079227 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.079315 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.079266 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.079315 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.079286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.079470 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.079372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z587j\" (UniqueName: \"kubernetes.io/projected/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-kube-api-access-z587j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181087 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181105 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181123 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181147 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z587j\" (UniqueName: \"kubernetes.io/projected/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-kube-api-access-z587j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181193 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181286 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181579 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181555 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181643 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181596 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.181714 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.181698 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.183707 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.183684 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.183911 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.183884 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.192316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.192296 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z587j\" (UniqueName: \"kubernetes.io/projected/34cce2ab-9093-42b7-b3ed-3ed7041b3e62-kube-api-access-z587j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tg89x\" (UID: \"34cce2ab-9093-42b7-b3ed-3ed7041b3e62\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.210072 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.210021 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vjjd7" Jul 14 15:58:28.308432 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.308355 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:28.465814 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:28.465785 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x"] Jul 14 15:58:28.468218 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:58:28.468190 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34cce2ab_9093_42b7_b3ed_3ed7041b3e62.slice/crio-51987bbaf273031cf4d05e66ee4dd8cc85766d37f887797d934b0e21266e2150 WatchSource:0}: Error finding container 51987bbaf273031cf4d05e66ee4dd8cc85766d37f887797d934b0e21266e2150: Status 404 returned error can't find the container with id 51987bbaf273031cf4d05e66ee4dd8cc85766d37f887797d934b0e21266e2150 Jul 14 15:58:29.204443 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:29.204407 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" event={"ID":"34cce2ab-9093-42b7-b3ed-3ed7041b3e62","Type":"ContainerStarted","Data":"c6ecbc4019cf96940dd0329520a851380bf6a6dde63f455399a995e7bf004f03"} Jul 14 15:58:29.204638 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:29.204472 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" event={"ID":"34cce2ab-9093-42b7-b3ed-3ed7041b3e62","Type":"ContainerStarted","Data":"51987bbaf273031cf4d05e66ee4dd8cc85766d37f887797d934b0e21266e2150"} Jul 14 15:58:29.227473 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:29.227421 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" podStartSLOduration=1.9704918 podStartE2EDuration="2.227404875s" totalImagesPullingTime="256.913075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:58:29.225183415 +0000 UTC m=+704.611270802" watchObservedRunningTime="2026-07-14 15:58:29.227404875 +0000 UTC m=+704.613492261" Jul 14 15:58:32.504225 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:32.504183 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:32.504937 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:32.504913 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:32.521256 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:32.521222 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:33.237865 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:33.237831 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h" Jul 14 15:58:38.309068 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:38.308947 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:38.309477 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:38.309071 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:38.321447 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:38.321421 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:39.263650 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:39.263620 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tg89x" Jul 14 15:58:43.493201 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.493168 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk"] Jul 14 15:58:43.500159 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.500132 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.502474 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.502451 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 14 15:58:43.513524 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.513487 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk"] Jul 14 15:58:43.519678 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.519648 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crcwn\" (UniqueName: \"kubernetes.io/projected/6a1a350c-d50b-4b60-9ca6-516039332ea3-kube-api-access-crcwn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.519817 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.519712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.519817 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.519760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a1a350c-d50b-4b60-9ca6-516039332ea3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.519817 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.519803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.519974 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.519833 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.519974 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.519940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621416 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crcwn\" (UniqueName: \"kubernetes.io/projected/6a1a350c-d50b-4b60-9ca6-516039332ea3-kube-api-access-crcwn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621614 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621441 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621614 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a1a350c-d50b-4b60-9ca6-516039332ea3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621614 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621614 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621540 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621614 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621582 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621916 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.621950 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621940 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.622130 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.621968 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.623828 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.623806 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a1a350c-d50b-4b60-9ca6-516039332ea3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.624209 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.624189 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a1a350c-d50b-4b60-9ca6-516039332ea3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.635649 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.635620 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crcwn\" (UniqueName: \"kubernetes.io/projected/6a1a350c-d50b-4b60-9ca6-516039332ea3-kube-api-access-crcwn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk\" (UID: \"6a1a350c-d50b-4b60-9ca6-516039332ea3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.820008 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.819917 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:43.962001 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:43.961973 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk"] Jul 14 15:58:43.964478 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:58:43.964454 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a1a350c_d50b_4b60_9ca6_516039332ea3.slice/crio-68372a52c4e2057dc0f6d9ea1e4b44db21066745627631755223db44b6114095 WatchSource:0}: Error finding container 68372a52c4e2057dc0f6d9ea1e4b44db21066745627631755223db44b6114095: Status 404 returned error can't find the container with id 68372a52c4e2057dc0f6d9ea1e4b44db21066745627631755223db44b6114095 Jul 14 15:58:44.277312 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:44.277281 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" event={"ID":"6a1a350c-d50b-4b60-9ca6-516039332ea3","Type":"ContainerStarted","Data":"68372a52c4e2057dc0f6d9ea1e4b44db21066745627631755223db44b6114095"} Jul 14 15:58:45.286739 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:45.286706 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" event={"ID":"6a1a350c-d50b-4b60-9ca6-516039332ea3","Type":"ContainerStarted","Data":"8e0ec117fc19f28b0c6905125e2ea9087148863f85a52f004fc673ecf02dbe9a"} Jul 14 15:58:45.305589 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:45.305536 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" podStartSLOduration=2.046982712 podStartE2EDuration="2.305519699s" totalImagesPullingTime="258.536987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:58:45.30389326 +0000 UTC m=+720.689980647" watchObservedRunningTime="2026-07-14 15:58:45.305519699 +0000 UTC m=+720.691607087" Jul 14 15:58:53.820810 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:53.820762 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:53.821336 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:53.820832 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:53.836316 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:53.836287 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:58:54.344465 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:58:54.344426 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk" Jul 14 15:59:46.613858 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.613815 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d9bddb874-6b7gk"] Jul 14 15:59:46.617290 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.617268 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.625311 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.625282 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d9bddb874-6b7gk"] Jul 14 15:59:46.697030 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.696991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9hpt\" (UniqueName: \"kubernetes.io/projected/af727704-b79b-4707-9d18-f7680f64c52f-kube-api-access-t9hpt\") pod \"authorino-d9bddb874-6b7gk\" (UID: \"af727704-b79b-4707-9d18-f7680f64c52f\") " pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.697260 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.697069 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/af727704-b79b-4707-9d18-f7680f64c52f-tls-cert\") pod \"authorino-d9bddb874-6b7gk\" (UID: \"af727704-b79b-4707-9d18-f7680f64c52f\") " pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.797725 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.797682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/af727704-b79b-4707-9d18-f7680f64c52f-tls-cert\") pod \"authorino-d9bddb874-6b7gk\" (UID: \"af727704-b79b-4707-9d18-f7680f64c52f\") " pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.797904 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.797831 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9hpt\" (UniqueName: \"kubernetes.io/projected/af727704-b79b-4707-9d18-f7680f64c52f-kube-api-access-t9hpt\") pod \"authorino-d9bddb874-6b7gk\" (UID: \"af727704-b79b-4707-9d18-f7680f64c52f\") " pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.800393 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.800360 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/af727704-b79b-4707-9d18-f7680f64c52f-tls-cert\") pod \"authorino-d9bddb874-6b7gk\" (UID: \"af727704-b79b-4707-9d18-f7680f64c52f\") " pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.806434 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.806407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9hpt\" (UniqueName: \"kubernetes.io/projected/af727704-b79b-4707-9d18-f7680f64c52f-kube-api-access-t9hpt\") pod \"authorino-d9bddb874-6b7gk\" (UID: \"af727704-b79b-4707-9d18-f7680f64c52f\") " pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:46.927825 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:46.927790 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d9bddb874-6b7gk" Jul 14 15:59:47.078238 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:47.078208 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d9bddb874-6b7gk"] Jul 14 15:59:47.080213 ip-10-0-139-147 kubenswrapper[2573]: W0714 15:59:47.080189 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf727704_b79b_4707_9d18_f7680f64c52f.slice/crio-fe19b39cd4bcbb31319390aa680c5512e1ea27f70542ea0521620a0917526d8e WatchSource:0}: Error finding container fe19b39cd4bcbb31319390aa680c5512e1ea27f70542ea0521620a0917526d8e: Status 404 returned error can't find the container with id fe19b39cd4bcbb31319390aa680c5512e1ea27f70542ea0521620a0917526d8e Jul 14 15:59:47.587972 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:47.587943 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d9bddb874-6b7gk" event={"ID":"af727704-b79b-4707-9d18-f7680f64c52f","Type":"ContainerStarted","Data":"fe19b39cd4bcbb31319390aa680c5512e1ea27f70542ea0521620a0917526d8e"} Jul 14 15:59:48.596238 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:48.596194 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d9bddb874-6b7gk" event={"ID":"af727704-b79b-4707-9d18-f7680f64c52f","Type":"ContainerStarted","Data":"dac3e8a0a0c8ea58938d5f9f7ed22f241f3d0eef4a059b261779b243ef94013f"} Jul 14 15:59:48.615604 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:48.615551 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d9bddb874-6b7gk" podStartSLOduration=2.151311937 podStartE2EDuration="2.615534661s" totalImagesPullingTime="464.222724ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:59:48.613409092 +0000 UTC m=+783.999496479" watchObservedRunningTime="2026-07-14 15:59:48.615534661 +0000 UTC m=+784.001622047" Jul 14 15:59:48.648476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:48.647844 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6df6ff8cf9-dbftd"] Jul 14 15:59:48.648476 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:48.648097 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" podUID="97775eb3-ce21-457d-b721-af3a602aa5ce" containerName="authorino" containerID="cri-o://eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1" gracePeriod=30 Jul 14 15:59:48.910025 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:48.910002 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:59:49.016408 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.016360 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7q7j\" (UniqueName: \"kubernetes.io/projected/97775eb3-ce21-457d-b721-af3a602aa5ce-kube-api-access-v7q7j\") pod \"97775eb3-ce21-457d-b721-af3a602aa5ce\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " Jul 14 15:59:49.016588 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.016509 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97775eb3-ce21-457d-b721-af3a602aa5ce-tls-cert\") pod \"97775eb3-ce21-457d-b721-af3a602aa5ce\" (UID: \"97775eb3-ce21-457d-b721-af3a602aa5ce\") " Jul 14 15:59:49.018668 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.018627 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97775eb3-ce21-457d-b721-af3a602aa5ce-kube-api-access-v7q7j" (OuterVolumeSpecName: "kube-api-access-v7q7j") pod "97775eb3-ce21-457d-b721-af3a602aa5ce" (UID: "97775eb3-ce21-457d-b721-af3a602aa5ce"). InnerVolumeSpecName "kube-api-access-v7q7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 15:59:49.029343 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.029304 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97775eb3-ce21-457d-b721-af3a602aa5ce-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "97775eb3-ce21-457d-b721-af3a602aa5ce" (UID: "97775eb3-ce21-457d-b721-af3a602aa5ce"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 15:59:49.118177 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.118143 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7q7j\" (UniqueName: \"kubernetes.io/projected/97775eb3-ce21-457d-b721-af3a602aa5ce-kube-api-access-v7q7j\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:59:49.118177 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.118170 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97775eb3-ce21-457d-b721-af3a602aa5ce-tls-cert\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 15:59:49.602777 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.602743 2573 generic.go:356] "Generic (PLEG): container finished" podID="97775eb3-ce21-457d-b721-af3a602aa5ce" containerID="eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1" exitCode=0 Jul 14 15:59:49.602777 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.602777 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" Jul 14 15:59:49.603264 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.602831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" event={"ID":"97775eb3-ce21-457d-b721-af3a602aa5ce","Type":"ContainerDied","Data":"eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1"} Jul 14 15:59:49.603264 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.602881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df6ff8cf9-dbftd" event={"ID":"97775eb3-ce21-457d-b721-af3a602aa5ce","Type":"ContainerDied","Data":"776b7beb503e1f5ea57c2900f1748737b6f4fd9b476e8d94059f4a7ce35909cc"} Jul 14 15:59:49.603264 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.602904 2573 scope.go:117] "RemoveContainer" containerID="eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1" Jul 14 15:59:49.613442 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.613422 2573 scope.go:117] "RemoveContainer" containerID="eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1" Jul 14 15:59:49.613708 ip-10-0-139-147 kubenswrapper[2573]: E0714 15:59:49.613686 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1\": container with ID starting with eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1 not found: ID does not exist" containerID="eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1" Jul 14 15:59:49.613810 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.613713 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1"} err="failed to get container status \"eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1\": rpc error: code = NotFound desc = could not find container \"eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1\": container with ID starting with eedd8b559078e3fb7320c975a7378d4948958f8979ddfacb00a23c1ef0bf1ec1 not found: ID does not exist" Jul 14 15:59:49.619330 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.619307 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6df6ff8cf9-dbftd"] Jul 14 15:59:49.626981 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:49.626955 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6df6ff8cf9-dbftd"] Jul 14 15:59:51.099760 ip-10-0-139-147 kubenswrapper[2573]: I0714 15:59:51.099721 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97775eb3-ce21-457d-b721-af3a602aa5ce" path="/var/lib/kubelet/pods/97775eb3-ce21-457d-b721-af3a602aa5ce/volumes" Jul 14 16:01:13.091347 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.091314 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c4db64696-q2kpl"] Jul 14 16:01:13.091878 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.091514 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7c4db64696-q2kpl" podUID="a0eec778-0478-4a1f-9203-edc6f41d1eed" containerName="manager" containerID="cri-o://ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e" gracePeriod=30 Jul 14 16:01:13.345129 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.345058 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 16:01:13.417888 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.417855 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0eec778-0478-4a1f-9203-edc6f41d1eed-webhook-cert\") pod \"a0eec778-0478-4a1f-9203-edc6f41d1eed\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " Jul 14 16:01:13.417888 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.417895 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp24s\" (UniqueName: \"kubernetes.io/projected/a0eec778-0478-4a1f-9203-edc6f41d1eed-kube-api-access-tp24s\") pod \"a0eec778-0478-4a1f-9203-edc6f41d1eed\" (UID: \"a0eec778-0478-4a1f-9203-edc6f41d1eed\") " Jul 14 16:01:13.420252 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.420213 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0eec778-0478-4a1f-9203-edc6f41d1eed-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a0eec778-0478-4a1f-9203-edc6f41d1eed" (UID: "a0eec778-0478-4a1f-9203-edc6f41d1eed"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 16:01:13.420252 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.420215 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0eec778-0478-4a1f-9203-edc6f41d1eed-kube-api-access-tp24s" (OuterVolumeSpecName: "kube-api-access-tp24s") pod "a0eec778-0478-4a1f-9203-edc6f41d1eed" (UID: "a0eec778-0478-4a1f-9203-edc6f41d1eed"). InnerVolumeSpecName "kube-api-access-tp24s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 16:01:13.519215 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.519176 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0eec778-0478-4a1f-9203-edc6f41d1eed-webhook-cert\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:01:13.519215 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:13.519209 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp24s\" (UniqueName: \"kubernetes.io/projected/a0eec778-0478-4a1f-9203-edc6f41d1eed-kube-api-access-tp24s\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:01:14.021200 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.021154 2573 generic.go:356] "Generic (PLEG): container finished" podID="a0eec778-0478-4a1f-9203-edc6f41d1eed" containerID="ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e" exitCode=0 Jul 14 16:01:14.021383 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.021210 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c4db64696-q2kpl" Jul 14 16:01:14.021383 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.021246 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c4db64696-q2kpl" event={"ID":"a0eec778-0478-4a1f-9203-edc6f41d1eed","Type":"ContainerDied","Data":"ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e"} Jul 14 16:01:14.021383 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.021334 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c4db64696-q2kpl" event={"ID":"a0eec778-0478-4a1f-9203-edc6f41d1eed","Type":"ContainerDied","Data":"7e2fc6e12d4c1c1451bba598e489a8da9a2c82761530387793371570559ddd0b"} Jul 14 16:01:14.021383 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.021360 2573 scope.go:117] "RemoveContainer" containerID="ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e" Jul 14 16:01:14.030668 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.030648 2573 scope.go:117] "RemoveContainer" containerID="ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e" Jul 14 16:01:14.030988 ip-10-0-139-147 kubenswrapper[2573]: E0714 16:01:14.030965 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e\": container with ID starting with ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e not found: ID does not exist" containerID="ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e" Jul 14 16:01:14.031080 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.031004 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e"} err="failed to get container status \"ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e\": rpc error: code = NotFound desc = could not find container \"ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e\": container with ID starting with ec710ffd7a4a6cfe3acfa909876d043d01d130e1551a9d406d8ee0c5b576020e not found: ID does not exist" Jul 14 16:01:14.043149 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.043121 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c4db64696-q2kpl"] Jul 14 16:01:14.047087 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:14.047064 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c4db64696-q2kpl"] Jul 14 16:01:15.099087 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:15.099028 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0eec778-0478-4a1f-9203-edc6f41d1eed" path="/var/lib/kubelet/pods/a0eec778-0478-4a1f-9203-edc6f41d1eed/volumes" Jul 14 16:01:45.059611 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:45.059530 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:01:45.063232 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:01:45.063209 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:06:19.848643 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:19.848552 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq"] Jul 14 16:06:19.849222 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:19.848809 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" podUID="c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" containerName="manager" containerID="cri-o://0badff99b542d7b446d04742c7de60b9e87688d4c023b9a86b4629ba788a95fa" gracePeriod=10 Jul 14 16:06:23.494773 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.494739 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" containerID="0badff99b542d7b446d04742c7de60b9e87688d4c023b9a86b4629ba788a95fa" exitCode=0 Jul 14 16:06:23.495122 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.494815 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" event={"ID":"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd","Type":"ContainerDied","Data":"0badff99b542d7b446d04742c7de60b9e87688d4c023b9a86b4629ba788a95fa"} Jul 14 16:06:23.495122 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.494875 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" event={"ID":"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd","Type":"ContainerDied","Data":"7bf83fc63a9feb68af410eb963d6d89d842a80289cf8ee9230e7e188d1ca5ae6"} Jul 14 16:06:23.495122 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.494891 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bf83fc63a9feb68af410eb963d6d89d842a80289cf8ee9230e7e188d1ca5ae6" Jul 14 16:06:23.507361 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.507337 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 16:06:23.606835 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.606748 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-extensions-socket-volume\") pod \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " Jul 14 16:06:23.606974 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.606838 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdht6\" (UniqueName: \"kubernetes.io/projected/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-kube-api-access-gdht6\") pod \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\" (UID: \"c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd\") " Jul 14 16:06:23.607192 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.607169 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" (UID: "c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 16:06:23.608924 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.608899 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-kube-api-access-gdht6" (OuterVolumeSpecName: "kube-api-access-gdht6") pod "c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" (UID: "c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd"). InnerVolumeSpecName "kube-api-access-gdht6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 16:06:23.708291 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.708250 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-extensions-socket-volume\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:06:23.708291 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:23.708285 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdht6\" (UniqueName: \"kubernetes.io/projected/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd-kube-api-access-gdht6\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:06:24.499490 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:24.499456 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq" Jul 14 16:06:24.522602 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:24.522571 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq"] Jul 14 16:06:24.526442 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:24.526419 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qpvtq"] Jul 14 16:06:25.100321 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:25.100291 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" path="/var/lib/kubelet/pods/c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd/volumes" Jul 14 16:06:45.089932 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:45.089902 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:06:45.098957 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:45.098923 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:06:45.157349 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:06:45.157319 2573 scope.go:117] "RemoveContainer" containerID="0badff99b542d7b446d04742c7de60b9e87688d4c023b9a86b4629ba788a95fa" Jul 14 16:07:21.034511 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.034469 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq"] Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035067 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97775eb3-ce21-457d-b721-af3a602aa5ce" containerName="authorino" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035087 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="97775eb3-ce21-457d-b721-af3a602aa5ce" containerName="authorino" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035099 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0eec778-0478-4a1f-9203-edc6f41d1eed" containerName="manager" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035106 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0eec778-0478-4a1f-9203-edc6f41d1eed" containerName="manager" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035135 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" containerName="manager" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035143 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" containerName="manager" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035251 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ff41c7-ba7c-4e6f-a1b2-3d3eef77a4bd" containerName="manager" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035267 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="97775eb3-ce21-457d-b721-af3a602aa5ce" containerName="authorino" Jul 14 16:07:21.035372 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.035276 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0eec778-0478-4a1f-9203-edc6f41d1eed" containerName="manager" Jul 14 16:07:21.038368 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.038346 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.040977 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.040959 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zqvcz\"" Jul 14 16:07:21.050716 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.050681 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq"] Jul 14 16:07:21.141270 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.141234 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7vrw\" (UniqueName: \"kubernetes.io/projected/21d45212-b2de-4ecd-9214-08a50d603aad-kube-api-access-h7vrw\") pod \"kuadrant-operator-controller-manager-86878c769f-rrrbq\" (UID: \"21d45212-b2de-4ecd-9214-08a50d603aad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.141444 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.141342 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d45212-b2de-4ecd-9214-08a50d603aad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrrbq\" (UID: \"21d45212-b2de-4ecd-9214-08a50d603aad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.242608 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.242557 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d45212-b2de-4ecd-9214-08a50d603aad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrrbq\" (UID: \"21d45212-b2de-4ecd-9214-08a50d603aad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.242749 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.242626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7vrw\" (UniqueName: \"kubernetes.io/projected/21d45212-b2de-4ecd-9214-08a50d603aad-kube-api-access-h7vrw\") pod \"kuadrant-operator-controller-manager-86878c769f-rrrbq\" (UID: \"21d45212-b2de-4ecd-9214-08a50d603aad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.242950 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.242930 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d45212-b2de-4ecd-9214-08a50d603aad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrrbq\" (UID: \"21d45212-b2de-4ecd-9214-08a50d603aad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.253690 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.253653 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7vrw\" (UniqueName: \"kubernetes.io/projected/21d45212-b2de-4ecd-9214-08a50d603aad-kube-api-access-h7vrw\") pod \"kuadrant-operator-controller-manager-86878c769f-rrrbq\" (UID: \"21d45212-b2de-4ecd-9214-08a50d603aad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.348373 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.348291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.490636 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.490610 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq"] Jul 14 16:07:21.493760 ip-10-0-139-147 kubenswrapper[2573]: W0714 16:07:21.493732 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d45212_b2de_4ecd_9214_08a50d603aad.slice/crio-e672123c926e04ef5566f068ee7d27e6cf4523558a70742b673917efd01d9e55 WatchSource:0}: Error finding container e672123c926e04ef5566f068ee7d27e6cf4523558a70742b673917efd01d9e55: Status 404 returned error can't find the container with id e672123c926e04ef5566f068ee7d27e6cf4523558a70742b673917efd01d9e55 Jul 14 16:07:21.768101 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.768032 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" event={"ID":"21d45212-b2de-4ecd-9214-08a50d603aad","Type":"ContainerStarted","Data":"018a2e9d835a3b323424a80dc4a1066dc2fa652ace8725df9a703847e064c215"} Jul 14 16:07:21.768285 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.768135 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" event={"ID":"21d45212-b2de-4ecd-9214-08a50d603aad","Type":"ContainerStarted","Data":"e672123c926e04ef5566f068ee7d27e6cf4523558a70742b673917efd01d9e55"} Jul 14 16:07:21.768285 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.768163 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:07:21.789309 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:21.789243 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" podStartSLOduration=0.789220196 podStartE2EDuration="789.220196ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:07:21.787389301 +0000 UTC m=+1237.173476688" watchObservedRunningTime="2026-07-14 16:07:21.789220196 +0000 UTC m=+1237.175307585" Jul 14 16:07:32.775589 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:07:32.775553 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrrbq" Jul 14 16:11:45.125307 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:11:45.125278 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:11:45.130459 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:11:45.130431 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:16:45.154219 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:16:45.154189 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:16:45.161339 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:16:45.161317 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:21:45.183123 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:21:45.183089 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:21:45.190096 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:21:45.190072 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:26:45.211358 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:26:45.211328 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:26:45.218482 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:26:45.218459 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:31:45.240575 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:31:45.240481 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:31:45.257955 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:31:45.257926 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:32:58.801068 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.801013 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx"] Jul 14 16:32:58.804780 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.804759 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.808224 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.808200 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"openshift-service-ca.crt\"" Jul 14 16:32:58.808360 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.808275 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"default-dockercfg-sf92j\"" Jul 14 16:32:58.808429 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.808211 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"kube-root-ca.crt\"" Jul 14 16:32:58.808429 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.808200 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"sub-test-model-21b35e-kserve-self-signed-certs\"" Jul 14 16:32:58.816137 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.816112 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx"] Jul 14 16:32:58.840544 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.840509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-dshm\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.840544 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.840545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-model-cache\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.840758 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.840573 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-tmp-dir\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.840758 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.840669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-home\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.840758 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.840703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de37d34-5b67-472c-ace4-5ecafe07a51f-tls-certs\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.840758 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.840743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fhmg\" (UniqueName: \"kubernetes.io/projected/3de37d34-5b67-472c-ace4-5ecafe07a51f-kube-api-access-6fhmg\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.941895 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.941853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fhmg\" (UniqueName: \"kubernetes.io/projected/3de37d34-5b67-472c-ace4-5ecafe07a51f-kube-api-access-6fhmg\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942131 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.941921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-dshm\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942131 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.941941 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-model-cache\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942131 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.941972 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-tmp-dir\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942131 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.942002 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-home\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942131 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.942021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de37d34-5b67-472c-ace4-5ecafe07a51f-tls-certs\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942430 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.942404 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-model-cache\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942504 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.942483 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-tmp-dir\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.942559 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.942514 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-home\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.944477 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.944448 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-dshm\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.944590 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.944568 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de37d34-5b67-472c-ace4-5ecafe07a51f-tls-certs\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:58.951588 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:58.951566 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fhmg\" (UniqueName: \"kubernetes.io/projected/3de37d34-5b67-472c-ace4-5ecafe07a51f-kube-api-access-6fhmg\") pod \"sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:59.115295 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:59.115197 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:32:59.268819 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:32:59.268792 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx"] Jul 14 16:32:59.271387 ip-10-0-139-147 kubenswrapper[2573]: W0714 16:32:59.271348 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3de37d34_5b67_472c_ace4_5ecafe07a51f.slice/crio-a7dc8b56492fc6c343bcecbccb6521d74a0d5506dde7bf5b4e0ec84e0c6f49f0 WatchSource:0}: Error finding container a7dc8b56492fc6c343bcecbccb6521d74a0d5506dde7bf5b4e0ec84e0c6f49f0: Status 404 returned error can't find the container with id a7dc8b56492fc6c343bcecbccb6521d74a0d5506dde7bf5b4e0ec84e0c6f49f0 Jul 14 16:33:00.217512 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:00.217476 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" event={"ID":"3de37d34-5b67-472c-ace4-5ecafe07a51f","Type":"ContainerStarted","Data":"98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58"} Jul 14 16:33:00.217906 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:00.217525 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" event={"ID":"3de37d34-5b67-472c-ace4-5ecafe07a51f","Type":"ContainerStarted","Data":"a7dc8b56492fc6c343bcecbccb6521d74a0d5506dde7bf5b4e0ec84e0c6f49f0"} Jul 14 16:33:00.241375 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:00.241330 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" podStartSLOduration=2.241314146 podStartE2EDuration="2.241314146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:33:00.239423809 +0000 UTC m=+2775.625511195" watchObservedRunningTime="2026-07-14 16:33:00.241314146 +0000 UTC m=+2775.627401532" Jul 14 16:33:09.115907 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:09.115822 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:33:09.116402 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:09.116076 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:33:09.128451 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:09.128420 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:33:09.274083 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:33:09.274036 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:34:08.111190 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:08.111151 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx"] Jul 14 16:34:23.132305 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.132232 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" podUID="3de37d34-5b67-472c-ace4-5ecafe07a51f" containerName="main" containerID="cri-o://98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58" gracePeriod=45 Jul 14 16:34:23.388765 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.388741 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:34:23.434691 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.434650 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de37d34-5b67-472c-ace4-5ecafe07a51f-tls-certs\") pod \"3de37d34-5b67-472c-ace4-5ecafe07a51f\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " Jul 14 16:34:23.434874 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.434744 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-model-cache\") pod \"3de37d34-5b67-472c-ace4-5ecafe07a51f\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " Jul 14 16:34:23.434874 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.434797 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fhmg\" (UniqueName: \"kubernetes.io/projected/3de37d34-5b67-472c-ace4-5ecafe07a51f-kube-api-access-6fhmg\") pod \"3de37d34-5b67-472c-ace4-5ecafe07a51f\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " Jul 14 16:34:23.434874 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.434857 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-dshm\") pod \"3de37d34-5b67-472c-ace4-5ecafe07a51f\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " Jul 14 16:34:23.435065 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.434886 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-home\") pod \"3de37d34-5b67-472c-ace4-5ecafe07a51f\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " Jul 14 16:34:23.435065 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.434918 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-tmp-dir\") pod \"3de37d34-5b67-472c-ace4-5ecafe07a51f\" (UID: \"3de37d34-5b67-472c-ace4-5ecafe07a51f\") " Jul 14 16:34:23.435190 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.435081 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-model-cache" (OuterVolumeSpecName: "model-cache") pod "3de37d34-5b67-472c-ace4-5ecafe07a51f" (UID: "3de37d34-5b67-472c-ace4-5ecafe07a51f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 16:34:23.435358 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.435289 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3de37d34-5b67-472c-ace4-5ecafe07a51f" (UID: "3de37d34-5b67-472c-ace4-5ecafe07a51f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 16:34:23.435358 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.435310 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-home" (OuterVolumeSpecName: "home") pod "3de37d34-5b67-472c-ace4-5ecafe07a51f" (UID: "3de37d34-5b67-472c-ace4-5ecafe07a51f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 16:34:23.435358 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.435299 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-model-cache\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:34:23.437636 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.437596 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-dshm" (OuterVolumeSpecName: "dshm") pod "3de37d34-5b67-472c-ace4-5ecafe07a51f" (UID: "3de37d34-5b67-472c-ace4-5ecafe07a51f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 16:34:23.437751 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.437701 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3de37d34-5b67-472c-ace4-5ecafe07a51f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3de37d34-5b67-472c-ace4-5ecafe07a51f" (UID: "3de37d34-5b67-472c-ace4-5ecafe07a51f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 16:34:23.437811 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.437764 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3de37d34-5b67-472c-ace4-5ecafe07a51f-kube-api-access-6fhmg" (OuterVolumeSpecName: "kube-api-access-6fhmg") pod "3de37d34-5b67-472c-ace4-5ecafe07a51f" (UID: "3de37d34-5b67-472c-ace4-5ecafe07a51f"). InnerVolumeSpecName "kube-api-access-6fhmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 16:34:23.536632 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.536587 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3de37d34-5b67-472c-ace4-5ecafe07a51f-tls-certs\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:34:23.536632 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.536627 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fhmg\" (UniqueName: \"kubernetes.io/projected/3de37d34-5b67-472c-ace4-5ecafe07a51f-kube-api-access-6fhmg\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:34:23.536632 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.536639 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-dshm\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:34:23.536873 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.536651 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-home\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:34:23.536873 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.536665 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3de37d34-5b67-472c-ace4-5ecafe07a51f-tmp-dir\") on node \"ip-10-0-139-147.ec2.internal\" DevicePath \"\"" Jul 14 16:34:23.623705 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.623669 2573 generic.go:356] "Generic (PLEG): container finished" podID="3de37d34-5b67-472c-ace4-5ecafe07a51f" containerID="98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58" exitCode=0 Jul 14 16:34:23.623888 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.623716 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" Jul 14 16:34:23.623888 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.623749 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" event={"ID":"3de37d34-5b67-472c-ace4-5ecafe07a51f","Type":"ContainerDied","Data":"98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58"} Jul 14 16:34:23.623888 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.623794 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx" event={"ID":"3de37d34-5b67-472c-ace4-5ecafe07a51f","Type":"ContainerDied","Data":"a7dc8b56492fc6c343bcecbccb6521d74a0d5506dde7bf5b4e0ec84e0c6f49f0"} Jul 14 16:34:23.623888 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.623817 2573 scope.go:117] "RemoveContainer" containerID="98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58" Jul 14 16:34:23.633761 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.633737 2573 scope.go:117] "RemoveContainer" containerID="98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58" Jul 14 16:34:23.634086 ip-10-0-139-147 kubenswrapper[2573]: E0714 16:34:23.634064 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58\": container with ID starting with 98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58 not found: ID does not exist" containerID="98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58" Jul 14 16:34:23.634157 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.634097 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58"} err="failed to get container status \"98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58\": rpc error: code = NotFound desc = could not find container \"98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58\": container with ID starting with 98eecfa5ead88fcd766923129bd5d7ea56c5065be60c11809e71d7695dfa0a58 not found: ID does not exist" Jul 14 16:34:23.646583 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.646510 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx"] Jul 14 16:34:23.650359 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:23.650338 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/sub-test-model-21b35e-kserve-85f4ccbf55-vqhgx"] Jul 14 16:34:25.100122 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:34:25.100081 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3de37d34-5b67-472c-ace4-5ecafe07a51f" path="/var/lib/kubelet/pods/3de37d34-5b67-472c-ace4-5ecafe07a51f/volumes" Jul 14 16:36:45.277736 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:36:45.277703 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:36:45.288054 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:36:45.288015 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bffbw_00d6d8d2-df0c-4093-a8c0-201d3ef36d37/ovn-acl-logging/0.log" Jul 14 16:37:17.713164 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:17.713132 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d9bddb874-6b7gk_af727704-b79b-4707-9d18-f7680f64c52f/authorino/0.log" Jul 14 16:37:23.088149 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:23.088116 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c8qtg_9e620cf1-9864-4e84-8c69-3b4e2fa93583/manager/0.log" Jul 14 16:37:23.213008 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:23.212958 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-52rmx_81b16e68-4d80-4cf2-b263-71eac1111b94/manager/0.log" Jul 14 16:37:23.456132 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:23.456099 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-g52jn_f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b/server/0.log" Jul 14 16:37:23.579071 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:23.579013 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n88b6_8caf40ed-7cb1-451b-9f94-f4081b24f9ce/manager/0.log" Jul 14 16:37:23.838352 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:23.838255 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-j4xqz_ebd8ad92-e085-4264-9d16-687d7140b148/copy-manifests/0.log" Jul 14 16:37:23.853507 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:23.853475 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-j4xqz_ebd8ad92-e085-4264-9d16-687d7140b148/manager/0.log" Jul 14 16:37:24.363334 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:24.363307 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6d7f74d569-6npt7_36f8b539-eb72-4d7c-a407-434c3323c007/maas-api/0.log" Jul 14 16:37:26.299971 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:26.299937 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zccxf_d1c613f9-54e8-4209-b515-e1d04759a91f/postgres/0.log" Jul 14 16:37:27.769153 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:27.769123 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d9bddb874-6b7gk_af727704-b79b-4707-9d18-f7680f64c52f/authorino/0.log" Jul 14 16:37:27.918317 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:27.918277 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8p26s_67a6c273-40ab-41db-8cba-ccbe115c5a89/manager/0.log" Jul 14 16:37:28.266615 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:28.266583 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6slqv_b4b50c86-cd5f-4bef-90d5-d6155056a1dc/registry-server/0.log" Jul 14 16:37:28.396016 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:28.395982 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rrrbq_21d45212-b2de-4ecd-9214-08a50d603aad/manager/0.log" Jul 14 16:37:29.405107 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:29.405035 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-t4vn5_0bdec269-762f-48a9-aa6b-0da3f21cf27f/kube-auth-proxy/0.log" Jul 14 16:37:29.534035 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:29.534006 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/1.log" Jul 14 16:37:29.771184 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:29.771098 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-zf6mx_c4dfdb8d-030b-4220-86d7-16e3126569ea/payload-processing/0.log" Jul 14 16:37:30.371283 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:30.371249 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-9wpms_286ff7c4-3fa3-4248-b61b-f8fcdcf2dfe6/main/0.log" Jul 14 16:37:30.488107 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:30.488080 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vjjd7_504f55b6-d22d-45e2-a8ea-ecc664881ece/main/0.log" Jul 14 16:37:30.605672 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:30.605641 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcrl9h_633ba653-b776-4c3f-a4f3-1dfd2a0725a5/main/0.log" Jul 14 16:37:30.723209 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:30.723181 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-tg89x_34cce2ab-9093-42b7-b3ed-3ed7041b3e62/main/0.log" Jul 14 16:37:30.840397 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:30.840366 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-qsmqk_6a1a350c-d50b-4b60-9ca6-516039332ea3/main/0.log" Jul 14 16:37:38.837019 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:38.836906 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bg8f6_1ba13aec-0ae1-4d1c-873e-e2ffba8b0da8/global-pull-secret-syncer/0.log" Jul 14 16:37:38.976499 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:38.976457 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gwrkq_8720cae8-3d43-472c-a510-7b8d5667da1d/konnectivity-agent/0.log" Jul 14 16:37:39.108300 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:39.108200 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-147.ec2.internal_9fa1b573b1bff3bbe3f49d2310b1a627/haproxy/0.log" Jul 14 16:37:44.209056 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:44.209012 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d9bddb874-6b7gk_af727704-b79b-4707-9d18-f7680f64c52f/authorino/0.log" Jul 14 16:37:44.238548 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:44.238511 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8p26s_67a6c273-40ab-41db-8cba-ccbe115c5a89/manager/0.log" Jul 14 16:37:44.347008 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:44.346980 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6slqv_b4b50c86-cd5f-4bef-90d5-d6155056a1dc/registry-server/0.log" Jul 14 16:37:44.473609 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:44.473524 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rrrbq_21d45212-b2de-4ecd-9214-08a50d603aad/manager/0.log" Jul 14 16:37:46.960438 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:46.960350 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nk82g_b03483f6-8589-4b95-947f-8379c4af2b8b/node-exporter/0.log" Jul 14 16:37:47.002334 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:47.002303 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nk82g_b03483f6-8589-4b95-947f-8379c4af2b8b/kube-rbac-proxy/0.log" Jul 14 16:37:47.077933 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:47.077906 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nk82g_b03483f6-8589-4b95-947f-8379c4af2b8b/init-textfile/0.log" Jul 14 16:37:49.111278 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:49.111248 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-srdcg_d680f3e0-33f9-430a-ad55-b8839d32415e/networking-console-plugin/0.log" Jul 14 16:37:50.429798 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.429762 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd"] Jul 14 16:37:50.430189 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.430146 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3de37d34-5b67-472c-ace4-5ecafe07a51f" containerName="main" Jul 14 16:37:50.430189 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.430157 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3de37d34-5b67-472c-ace4-5ecafe07a51f" containerName="main" Jul 14 16:37:50.430469 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.430240 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3de37d34-5b67-472c-ace4-5ecafe07a51f" containerName="main" Jul 14 16:37:50.433619 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.433599 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.435973 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.435953 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wmsrf\"/\"kube-root-ca.crt\"" Jul 14 16:37:50.436872 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.436853 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wmsrf\"/\"openshift-service-ca.crt\"" Jul 14 16:37:50.436872 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.436866 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wmsrf\"/\"default-dockercfg-pvx4s\"" Jul 14 16:37:50.441417 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.441390 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd"] Jul 14 16:37:50.499328 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.499290 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-proc\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.499328 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.499329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd5bs\" (UniqueName: \"kubernetes.io/projected/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-kube-api-access-wd5bs\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.499595 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.499376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-lib-modules\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.499595 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.499439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-podres\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.499595 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.499510 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-sys\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600646 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600605 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-podres\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600676 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-sys\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-proc\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600747 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd5bs\" (UniqueName: \"kubernetes.io/projected/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-kube-api-access-wd5bs\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-lib-modules\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600791 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-podres\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600809 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-proc\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.600842 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600795 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-sys\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.601141 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.600903 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-lib-modules\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.612489 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.612460 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd5bs\" (UniqueName: \"kubernetes.io/projected/040be7cc-e11b-4ec6-b6b0-76294a26bfe6-kube-api-access-wd5bs\") pod \"perf-node-gather-daemonset-tv8fd\" (UID: \"040be7cc-e11b-4ec6-b6b0-76294a26bfe6\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.744710 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.744603 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:50.883495 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:50.883291 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd"] Jul 14 16:37:50.885839 ip-10-0-139-147 kubenswrapper[2573]: W0714 16:37:50.885814 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod040be7cc_e11b_4ec6_b6b0_76294a26bfe6.slice/crio-afa64b6cbadaacf68a1bd79a429fd75dd6c4f8434b792039af5bb2772c69ea05 WatchSource:0}: Error finding container afa64b6cbadaacf68a1bd79a429fd75dd6c4f8434b792039af5bb2772c69ea05: Status 404 returned error can't find the container with id afa64b6cbadaacf68a1bd79a429fd75dd6c4f8434b792039af5bb2772c69ea05 Jul 14 16:37:51.621495 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:51.621454 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" event={"ID":"040be7cc-e11b-4ec6-b6b0-76294a26bfe6","Type":"ContainerStarted","Data":"702f962f6db090e1c2e4be809b668c82ab00c7e7ea23f070f38cbd714c0415b6"} Jul 14 16:37:51.621902 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:51.621529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" event={"ID":"040be7cc-e11b-4ec6-b6b0-76294a26bfe6","Type":"ContainerStarted","Data":"afa64b6cbadaacf68a1bd79a429fd75dd6c4f8434b792039af5bb2772c69ea05"} Jul 14 16:37:51.621902 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:51.621654 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:51.640656 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:51.640601 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" podStartSLOduration=1.640587322 podStartE2EDuration="1.640587322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 16:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:37:51.638868787 +0000 UTC m=+3067.024956175" watchObservedRunningTime="2026-07-14 16:37:51.640587322 +0000 UTC m=+3067.026674707" Jul 14 16:37:52.062620 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:52.062590 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d7652_ebcfae7d-263f-4a16-860a-1bc56f7ee88c/dns/0.log" Jul 14 16:37:52.096717 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:52.096686 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d7652_ebcfae7d-263f-4a16-860a-1bc56f7ee88c/kube-rbac-proxy/0.log" Jul 14 16:37:52.180024 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:52.179983 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g6vvg_bd82b629-05a6-4aa2-9a72-1c421ae37e0d/dns-node-resolver/0.log" Jul 14 16:37:52.849283 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:52.849250 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5bf6456885-v88g8_ffda826d-2c0c-49ad-9087-1dc19b74c902/registry/0.log" Jul 14 16:37:52.871812 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:52.871777 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hpv8z_ae7ada8b-e8d8-489d-bf68-b3b820771c1a/node-ca/0.log" Jul 14 16:37:54.114502 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:54.114465 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-t4vn5_0bdec269-762f-48a9-aa6b-0da3f21cf27f/kube-auth-proxy/0.log" Jul 14 16:37:54.144661 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:54.144607 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/0.log" Jul 14 16:37:54.175733 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:54.175692 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/1.log" Jul 14 16:37:54.237974 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:54.237943 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-zf6mx_c4dfdb8d-030b-4220-86d7-16e3126569ea/payload-processing/0.log" Jul 14 16:37:55.039825 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:55.039788 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sq9l9_d8da8bdc-83e2-4baf-a20d-fe118ad3541e/serve-healthcheck-canary/0.log" Jul 14 16:37:55.552145 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:55.552109 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4v2r5_6bdfe042-b0b5-4ebc-8c6d-8414d737327a/kube-rbac-proxy/0.log" Jul 14 16:37:55.583918 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:55.583894 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4v2r5_6bdfe042-b0b5-4ebc-8c6d-8414d737327a/exporter/0.log" Jul 14 16:37:55.607942 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:55.607917 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4v2r5_6bdfe042-b0b5-4ebc-8c6d-8414d737327a/extractor/0.log" Jul 14 16:37:57.637640 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:57.637614 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-tv8fd" Jul 14 16:37:58.018774 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.018691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c8qtg_9e620cf1-9864-4e84-8c69-3b4e2fa93583/manager/0.log" Jul 14 16:37:58.167877 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.167841 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-52rmx_81b16e68-4d80-4cf2-b263-71eac1111b94/manager/0.log" Jul 14 16:37:58.305224 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.305144 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-g52jn_f4b1b776-a609-4b8a-ae2f-cfa12aa8b41b/server/0.log" Jul 14 16:37:58.377585 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.377550 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n88b6_8caf40ed-7cb1-451b-9f94-f4081b24f9ce/manager/0.log" Jul 14 16:37:58.388094 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.388069 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-t4vn5_0bdec269-762f-48a9-aa6b-0da3f21cf27f/kube-auth-proxy/0.log" Jul 14 16:37:58.410753 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.410722 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/0.log" Jul 14 16:37:58.437774 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.437736 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/1.log" Jul 14 16:37:58.484922 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.484892 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-zf6mx_c4dfdb8d-030b-4220-86d7-16e3126569ea/payload-processing/0.log" Jul 14 16:37:58.520670 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.520632 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-j4xqz_ebd8ad92-e085-4264-9d16-687d7140b148/manager/0.log" Jul 14 16:37:58.540862 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:37:58.540836 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-j4xqz_ebd8ad92-e085-4264-9d16-687d7140b148/copy-manifests/0.log" Jul 14 16:38:00.095623 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:00.095596 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-t4vn5_0bdec269-762f-48a9-aa6b-0da3f21cf27f/kube-auth-proxy/0.log" Jul 14 16:38:00.121079 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:00.121029 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/0.log" Jul 14 16:38:00.152469 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:00.152442 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m2zw2_8864abe0-921f-41b4-91da-ac151bf2da49/istio-proxy/1.log" Jul 14 16:38:00.205710 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:00.205681 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-zf6mx_c4dfdb8d-030b-4220-86d7-16e3126569ea/payload-processing/0.log" Jul 14 16:38:05.550937 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:05.550905 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-gkrsx_4a02d65d-9638-4c31-adb2-f839a29bbbc2/migrator/0.log" Jul 14 16:38:05.584469 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:05.584433 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-gkrsx_4a02d65d-9638-4c31-adb2-f839a29bbbc2/graceful-termination/0.log" Jul 14 16:38:07.467451 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:07.467423 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hklg2_c581dd53-95b4-46a5-bd65-87722abac294/kube-multus-additional-cni-plugins/0.log" Jul 14 16:38:07.493220 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:07.493188 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hklg2_c581dd53-95b4-46a5-bd65-87722abac294/egress-router-binary-copy/0.log" Jul 14 16:38:07.518649 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:07.518620 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hklg2_c581dd53-95b4-46a5-bd65-87722abac294/cni-plugins/0.log" Jul 14 16:38:07.543670 ip-10-0-139-147 kubenswrapper[2573]: I0714 16:38:07.543639 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hklg2_c581dd53-95b4-46a5-bd65-87722abac294/bond-cni-plugin/0.log"