Jul 16 16:25:22.698864 ip-10-0-131-94 systemd[1]: Starting Kubernetes Kubelet... Jul 16 16:25:23.053224 ip-10-0-131-94 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 16:25:23.053224 ip-10-0-131-94 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 16:25:23.053224 ip-10-0-131-94 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 16:25:23.053224 ip-10-0-131-94 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 16:25:23.053224 ip-10-0-131-94 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 16:25:23.054094 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.054001 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 16:25:23.059109 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059084 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:25:23.059109 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059103 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:25:23.059109 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059109 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:25:23.059109 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059113 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:25:23.059109 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059117 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059121 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059125 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059129 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059132 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059139 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059145 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059150 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059154 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059158 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059162 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059165 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059169 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059172 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059177 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059181 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059184 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059188 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059193 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059198 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:25:23.059406 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059206 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059211 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059214 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059218 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059222 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059226 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059232 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059238 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059242 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059247 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059252 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059256 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059261 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059265 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059271 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059275 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059280 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059284 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059288 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:25:23.060254 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059292 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059297 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059301 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059305 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059310 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059315 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059319 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059323 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059328 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059332 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059336 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059341 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059346 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059350 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059355 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059359 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059364 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059369 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059374 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:25:23.061090 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059378 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059384 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059390 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059395 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059400 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059404 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059408 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059412 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059418 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059422 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059426 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059430 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059435 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059439 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059443 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059448 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059452 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059457 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059462 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059466 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:25:23.061943 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059470 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059474 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059479 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059483 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059487 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059491 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059495 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059499 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059503 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059508 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059512 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059516 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059520 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059524 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059546 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059552 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.059556 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060729 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060738 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060742 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:25:23.062804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060747 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060754 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060760 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060765 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060769 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060774 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060778 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060782 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060786 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060790 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060795 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060802 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060808 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060813 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060818 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060822 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060827 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060831 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060835 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:25:23.063648 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060839 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060844 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060848 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060852 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060856 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060862 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060866 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060870 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060874 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060879 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060884 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060888 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060892 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060896 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060900 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060906 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060910 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060914 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060918 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060922 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:25:23.064337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060926 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060931 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060935 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060939 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060944 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060949 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060953 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060957 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060961 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060965 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060970 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060974 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060978 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060982 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060986 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060991 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.060996 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061000 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061004 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061012 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:25:23.065006 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061018 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061023 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061028 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061033 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061038 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061042 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061046 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061051 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061055 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061059 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061064 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061068 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061072 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061076 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061081 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061084 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061088 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061097 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061101 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061105 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:25:23.065693 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061109 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061113 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061117 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061122 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061126 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061130 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061134 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061141 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061146 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061150 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061154 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061158 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061162 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061166 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061171 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061175 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061180 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061825 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061834 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:25:23.066180 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061839 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061844 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061848 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061852 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061857 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061861 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061865 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061869 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061873 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061877 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061882 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061887 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061891 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061895 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061899 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061903 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061907 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061911 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061915 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061919 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:25:23.066750 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061926 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061931 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061935 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061939 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061944 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061948 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061952 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061955 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061959 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061963 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061967 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061972 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061976 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061980 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061985 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061989 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061994 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.061998 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062002 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062007 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:25:23.067247 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062011 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062015 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062019 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062024 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062028 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062032 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062037 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062041 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062045 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062049 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062056 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062062 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062071 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062077 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062081 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062087 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062091 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062095 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062099 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:25:23.067793 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062103 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062107 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062112 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062116 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062120 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062124 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062128 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062132 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062139 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062144 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062150 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062154 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062159 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062163 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062167 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062171 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062175 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062180 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062186 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:25:23.068306 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062191 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062195 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062200 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062204 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062207 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062211 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062216 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062223 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062227 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062231 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062235 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062240 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062244 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062248 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062252 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062256 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062261 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062265 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.062270 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062378 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062390 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 16:25:23.068790 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062401 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062408 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062415 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062420 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062426 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062433 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062439 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062443 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062449 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062454 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062459 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062465 2560 flags.go:64] FLAG: --cgroup-root="" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062469 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062474 2560 flags.go:64] FLAG: --client-ca-file="" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062479 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062484 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062490 2560 flags.go:64] FLAG: --cluster-domain="" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062494 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062501 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062506 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062512 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062519 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062523 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 16:25:23.069295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062549 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062555 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062560 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062565 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062569 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062574 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062579 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062586 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062591 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062596 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062601 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062605 2560 flags.go:64] FLAG: --enable-server="true" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062610 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062618 2560 flags.go:64] FLAG: --event-burst="100" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062623 2560 flags.go:64] FLAG: --event-qps="50" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062628 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062633 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062638 2560 flags.go:64] FLAG: --eviction-hard="" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062644 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062649 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062655 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062660 2560 flags.go:64] FLAG: --eviction-soft="" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062664 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062669 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062674 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 16:25:23.069928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062678 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062683 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062690 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062695 2560 flags.go:64] FLAG: --feature-gates="" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062701 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062707 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062711 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062716 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062721 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062726 2560 flags.go:64] FLAG: --help="false" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062731 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062737 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062742 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062746 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062753 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062759 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062763 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062768 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062772 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062777 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062782 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062787 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062791 2560 flags.go:64] FLAG: --kube-reserved="" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062796 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 16:25:23.070582 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062801 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062805 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062809 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062814 2560 flags.go:64] FLAG: --lock-file="" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062819 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062824 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062828 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062838 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062843 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062849 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062856 2560 flags.go:64] FLAG: --logging-format="text" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062860 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062866 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062870 2560 flags.go:64] FLAG: --manifest-url="" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062875 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062882 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062887 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062893 2560 flags.go:64] FLAG: --max-pods="110" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062898 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062903 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062908 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062912 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062917 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062922 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062927 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 16:25:23.071182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062938 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062943 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062972 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062980 2560 flags.go:64] FLAG: --pod-cidr="" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062985 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062993 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.062998 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063003 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063008 2560 flags.go:64] FLAG: --port="10250" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063013 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063018 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08729fce527be5d66" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063023 2560 flags.go:64] FLAG: --qos-reserved="" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063029 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063034 2560 flags.go:64] FLAG: --register-node="true" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063038 2560 flags.go:64] FLAG: --register-with-taints="" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063045 2560 flags.go:64] FLAG: --registry-burst="10" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063049 2560 flags.go:64] FLAG: --registry-qps="5" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063057 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063062 2560 flags.go:64] FLAG: --reserved-memory="" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063068 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063075 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063081 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063085 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063090 2560 flags.go:64] FLAG: --runonce="false" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063094 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 16:25:23.071867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063100 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063105 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063109 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063114 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063119 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063124 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063130 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063134 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063139 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063144 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063149 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063154 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063158 2560 flags.go:64] FLAG: --system-cgroups="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063163 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063172 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063177 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063181 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063188 2560 flags.go:64] FLAG: --tls-min-version="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063192 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063197 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063202 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063207 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063212 2560 flags.go:64] FLAG: --v="2" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063218 2560 flags.go:64] FLAG: --version="false" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063230 2560 flags.go:64] FLAG: --vmodule="" Jul 16 16:25:23.072472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063236 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063241 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063427 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063434 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063439 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063444 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063449 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063453 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063458 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063463 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063467 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063471 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063475 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063480 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063484 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063488 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063493 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063497 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063501 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063505 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:25:23.073091 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063510 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063513 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063518 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063522 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063546 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063554 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063558 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063563 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063578 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063584 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063588 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063596 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063600 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063604 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063608 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063612 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063616 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063620 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063624 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:25:23.073612 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063628 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063633 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063637 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063641 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063645 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063649 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063654 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063658 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063662 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063666 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063670 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063674 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063678 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063682 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063686 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063690 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063694 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063699 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063703 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063707 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:25:23.074098 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063711 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063715 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063720 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063724 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063732 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063736 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063740 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063744 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063749 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063753 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063757 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063761 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063764 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063771 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063777 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063781 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063786 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063790 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063795 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:25:23.074617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063801 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063807 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063811 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063815 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063819 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063824 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063828 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063831 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063835 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063839 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063844 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063848 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063852 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063856 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063860 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063864 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063869 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063877 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063881 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063885 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:25:23.075087 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063889 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063893 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.063897 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.063906 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.071631 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.071647 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.071668 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.071677 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071751 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071758 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071762 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071765 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071768 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071771 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:25:23.075617 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071774 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071777 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071780 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071783 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071786 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071789 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071792 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071795 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071797 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071800 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071803 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071805 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071808 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071811 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071819 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071822 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071825 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071827 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071829 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071832 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:25:23.075969 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071834 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071837 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071839 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071842 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071845 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071848 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071850 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071853 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071855 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071859 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071863 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071865 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071868 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071871 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071874 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071876 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071879 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071883 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071886 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:25:23.076460 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071889 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071891 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071894 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071896 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071899 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071901 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071904 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071906 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071914 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071916 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071919 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071921 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071924 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071927 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071929 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071933 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071935 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071938 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071940 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:25:23.076962 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071946 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071949 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071951 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071954 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071957 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071959 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071962 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071964 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071967 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071969 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071972 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071974 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071977 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071979 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071981 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071984 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071986 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071989 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071991 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.071994 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:25:23.077468 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072003 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072005 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072014 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072016 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072019 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072022 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072025 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072027 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072030 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072033 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072036 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072038 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072041 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072043 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:25:23.077997 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072046 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.072051 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072199 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072204 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072206 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072209 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072212 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072214 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072217 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072220 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072222 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072225 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072227 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072230 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072232 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 16:25:23.078346 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072235 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072237 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072240 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072242 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072245 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072247 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072256 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072259 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072262 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072265 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072268 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072272 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072275 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072278 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072281 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072284 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072286 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072289 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072291 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 16:25:23.078736 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072294 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072296 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072299 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072301 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072303 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072306 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072308 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072311 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072313 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072316 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072318 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072321 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072323 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072326 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072328 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072332 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072335 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072338 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072340 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 16:25:23.079203 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072342 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072351 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072354 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072357 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072359 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072362 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072364 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072366 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072370 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072374 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072376 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072379 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072381 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072384 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072386 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072389 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072391 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072393 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072396 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072398 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 16:25:23.079682 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072401 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072403 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072406 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072408 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072411 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072413 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072415 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072418 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072420 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072423 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072426 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072428 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072431 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072433 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072441 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072444 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072447 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072450 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072452 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072455 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 16:25:23.080163 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072457 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072460 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072462 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072464 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072467 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072469 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072472 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:23.072474 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.072479 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.072625 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.075564 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.076881 2560 server.go:1019] "Starting client certificate rotation" Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.076991 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 16:25:23.081803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.077028 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 16:25:23.082165 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.081460 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 16:25:23.083173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.083158 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 16:25:23.092515 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.092497 2560 log.go:25] "Validated CRI v1 runtime API" Jul 16 16:25:23.095395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.095378 2560 log.go:25] "Validated CRI v1 image API" Jul 16 16:25:23.096222 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.096197 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 16:25:23.098501 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.098477 2560 fs.go:135] Filesystem UUIDs: map[0763e541-2e05-4623-bb31-4cb7ea1660d0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 dc144f84-0a82-4ad4-a595-ccea71491cf3:/dev/nvme0n1p3] Jul 16 16:25:23.098575 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.098503 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 16:25:23.104103 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.103988 2560 manager.go:216] Machine: {Timestamp:2026-07-16 16:25:23.102840589 +0000 UTC m=+0.297477497 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095105 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22ef741e94d102f69585a8d588d1d6 SystemUUID:ec22ef74-1e94-d102-f695-85a8d588d1d6 BootID:d2b857a2-9279-4425-aab4-eb70d4cd51d4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ed:66:58:7c:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ed:66:58:7c:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:5f:fc:14:9d:8a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 16:25:23.104103 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.104102 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 16:25:23.104199 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.104192 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 16:25:23.107063 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.107017 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 16:25:23.107226 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.107061 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 16:25:23.107986 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.107974 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 16:25:23.108029 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.107988 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 16:25:23.108029 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.107995 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 16:25:23.108515 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.108495 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 16:25:23.108708 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.108695 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 16:25:23.110161 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.110149 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 16:25:23.114045 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114030 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 16 16:25:23.114184 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114174 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 16:25:23.114220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114213 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 16 16:25:23.114250 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114223 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 16:25:23.114250 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114239 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 16:25:23.114307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114270 2560 kubelet.go:398] "Adding apiserver pod source" Jul 16 16:25:23.114307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114280 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 16:25:23.114734 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114721 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 16:25:23.114734 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.114743 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 16:25:23.118674 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.118502 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 16:25:23.120694 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120677 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 16:25:23.120757 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120711 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 16:25:23.120888 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120878 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 16:25:23.120919 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120892 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 16:25:23.120919 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120899 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 16:25:23.120919 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120905 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 16:25:23.120919 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120915 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 16:25:23.121020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.120921 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 16:25:23.121382 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.121372 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 16:25:23.121382 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.121381 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 16:25:23.121443 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.121389 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 16:25:23.121443 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.121394 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 16:25:23.121443 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.121403 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 16:25:23.121443 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.121411 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 16:25:23.122209 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.122197 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 16:25:23.122243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.122212 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 16:25:23.125232 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.125209 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 16:25:23.125322 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.125209 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 16:25:23.125322 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.125219 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-94.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 16:25:23.125757 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.125736 2560 server.go:1268] "Started kubelet" Jul 16 16:25:23.125854 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.125780 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 16:25:23.125998 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.125941 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 16:25:23.126088 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.126020 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 16:25:23.126676 ip-10-0-131-94 systemd[1]: Started Kubernetes Kubelet. Jul 16 16:25:23.127418 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.127402 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 16:25:23.127467 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.127438 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 16 16:25:23.131331 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.131312 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 16:25:23.132987 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.132120 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18c2d1d6db5d83e2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:ip-10-0-131-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-07-16 16:25:23.12568317 +0000 UTC m=+0.320320078,LastTimestamp:2026-07-16 16:25:23.12568317 +0000 UTC m=+0.320320078,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Jul 16 16:25:23.134420 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.134404 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 16:25:23.134482 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.134417 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 16:25:23.134751 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.134603 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.135119 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.135104 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 16:25:23.135508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.135107 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 16:25:23.135634 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.135617 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 16:25:23.135920 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.135758 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 16 16:25:23.135920 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.135765 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 16 16:25:23.136294 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136275 2560 factory.go:55] Registering systemd factory Jul 16 16:25:23.136367 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136297 2560 factory.go:223] Registration of the systemd container factory successfully Jul 16 16:25:23.136556 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136522 2560 factory.go:153] Registering CRI-O factory Jul 16 16:25:23.136642 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136559 2560 factory.go:223] Registration of the crio container factory successfully Jul 16 16:25:23.136642 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136626 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 16:25:23.136739 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136654 2560 factory.go:103] Registering Raw factory Jul 16 16:25:23.136739 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.136669 2560 manager.go:1196] Started watching for new ooms in manager Jul 16 16:25:23.137180 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.137163 2560 manager.go:349] Starting recovery of all containers Jul 16 16:25:23.139370 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.139335 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 16:25:23.139474 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.139386 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 16:25:23.150755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.150727 2560 manager.go:354] Recovery completed Jul 16 16:25:23.155586 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.155570 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.157858 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.157837 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.157927 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.157876 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.157927 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.157887 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.158393 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.158378 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 16:25:23.158393 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.158387 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.158393 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.158393 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 16:25:23.158560 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.158412 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 16 16:25:23.159100 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.159085 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.159168 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.159117 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.159168 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.159134 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.159720 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.159639 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18c2d1d6dd4881cf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:ip-10-0-131-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-94.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-07-16 16:25:23.157860815 +0000 UTC m=+0.352497728,LastTimestamp:2026-07-16 16:25:23.157860815 +0000 UTC m=+0.352497728,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Jul 16 16:25:23.160665 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.160651 2560 policy_none.go:49] "None policy: Start" Jul 16 16:25:23.160710 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.160671 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 16:25:23.160710 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.160685 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 16:25:23.162696 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.162684 2560 policy_none.go:47] "Start" Jul 16 16:25:23.167994 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.167970 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ch7z4" Jul 16 16:25:23.170617 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.170546 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18c2d1d6dd48d15e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:ip-10-0-131-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-94.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-07-16 16:25:23.157881182 +0000 UTC m=+0.352518090,LastTimestamp:2026-07-16 16:25:23.157881182 +0000 UTC m=+0.352518090,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Jul 16 16:25:23.175161 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.175055 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18c2d1d6dd48f81e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:ip-10-0-131-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-94.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-07-16 16:25:23.157891102 +0000 UTC m=+0.352528009,LastTimestamp:2026-07-16 16:25:23.157891102 +0000 UTC m=+0.352528009,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Jul 16 16:25:23.181092 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.181072 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ch7z4" Jul 16 16:25:23.182099 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.182031 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-94.ec2.internal.18c2d1d6dd4881cf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18c2d1d6dd4881cf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:ip-10-0-131-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-94.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-07-16 16:25:23.157860815 +0000 UTC m=+0.352497728,LastTimestamp:2026-07-16 16:25:23.159100518 +0000 UTC m=+0.353737429,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Jul 16 16:25:23.184250 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.184189 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-94.ec2.internal.18c2d1d6dd48d15e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18c2d1d6dd48d15e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:ip-10-0-131-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-94.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-07-16 16:25:23.157881182 +0000 UTC m=+0.352518090,LastTimestamp:2026-07-16 16:25:23.159123616 +0000 UTC m=+0.353760528,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Jul 16 16:25:23.203137 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203120 2560 manager.go:337] "Starting Device Plugin manager" Jul 16 16:25:23.203217 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.203161 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 16:25:23.203217 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203176 2560 server.go:87] "Starting device plugin registration server" Jul 16 16:25:23.203496 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203483 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 16:25:23.203558 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203500 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 16:25:23.203623 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203605 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 16:25:23.203776 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203755 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 16:25:23.203776 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.203777 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 16:25:23.204363 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.204261 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 16:25:23.204363 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.204301 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.261139 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.261092 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 16:25:23.263414 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.262389 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 16:25:23.263414 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.262427 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 16:25:23.263414 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.262479 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 16:25:23.263414 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.262512 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 16:25:23.265166 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.265145 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:25:23.304223 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.304123 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.305351 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.305325 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.305466 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.305359 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.305466 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.305371 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.305466 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.305396 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.315069 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.315050 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.315199 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.315076 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-94.ec2.internal\": node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.342077 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.342042 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.363621 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.363591 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal"] Jul 16 16:25:23.363776 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.363658 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.365114 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.365100 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.365191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.365141 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.365191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.365151 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.367504 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.367490 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.368020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368004 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.368110 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368043 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.368247 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368225 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.368322 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368254 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.368322 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368264 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.368758 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368744 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.368825 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368770 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.368825 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.368784 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.370469 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.370454 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.370551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.370480 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:23.371128 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.371111 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:23.371175 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.371141 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:23.371175 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.371153 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:23.393262 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.393237 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-94.ec2.internal\" not found" node="ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.397691 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.397675 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-94.ec2.internal\" not found" node="ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.437222 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.437192 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0aa522b1d98aef62f303d8039fdc50f5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"0aa522b1d98aef62f303d8039fdc50f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.437222 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.437222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0aa522b1d98aef62f303d8039fdc50f5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"0aa522b1d98aef62f303d8039fdc50f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.437382 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.437241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c01f07008c2f7f9b6faadbc640e87b1-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"3c01f07008c2f7f9b6faadbc640e87b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.442303 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.442288 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.538083 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.538049 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0aa522b1d98aef62f303d8039fdc50f5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"0aa522b1d98aef62f303d8039fdc50f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.538083 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.538085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0aa522b1d98aef62f303d8039fdc50f5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"0aa522b1d98aef62f303d8039fdc50f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.538286 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.538105 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c01f07008c2f7f9b6faadbc640e87b1-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"3c01f07008c2f7f9b6faadbc640e87b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.538286 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.538150 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0aa522b1d98aef62f303d8039fdc50f5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"0aa522b1d98aef62f303d8039fdc50f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.538286 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.538153 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c01f07008c2f7f9b6faadbc640e87b1-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"3c01f07008c2f7f9b6faadbc640e87b1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.538286 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.538154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0aa522b1d98aef62f303d8039fdc50f5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"0aa522b1d98aef62f303d8039fdc50f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.543105 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.543085 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.643985 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.643910 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.695114 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.695085 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.701100 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:23.701083 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Jul 16 16:25:23.744161 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.744123 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:23.844774 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:23.844729 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:25.331840 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:25.331810 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:25.432420 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:25.432375 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:25.462594 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.462552 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 16:25:25.463038 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.462709 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 16:25:25.532501 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:25.532471 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:25.571999 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.571898 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 16:20:24 +0000 UTC" deadline="2027-12-15 17:38:24.272671561 +0000 UTC" Jul 16 16:25:25.571999 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.571935 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12409h12m58.700741388s" Jul 16 16:25:25.574070 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.574041 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 16:25:25.575291 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.575273 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Jul 16 16:25:25.575478 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.575302 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 16:25:25.575478 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.575312 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Jul 16 16:25:25.633248 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:25.633220 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Jul 16 16:25:25.704970 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.704942 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:25:25.717676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.717638 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:25:25.722216 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.722195 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Jul 16 16:25:25.745403 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.745370 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 16:25:25.746979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.746963 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Jul 16 16:25:25.757756 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.757738 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 16:25:25.767876 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.767854 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:25:25.781077 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:25.781006 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa522b1d98aef62f303d8039fdc50f5.slice/crio-9d3f64079066706c9d7b24f2f4fb347f3a5b187f0b58c9c28e6d73f578c83972 WatchSource:0}: Error finding container 9d3f64079066706c9d7b24f2f4fb347f3a5b187f0b58c9c28e6d73f578c83972: Status 404 returned error can't find the container with id 9d3f64079066706c9d7b24f2f4fb347f3a5b187f0b58c9c28e6d73f578c83972 Jul 16 16:25:25.781337 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:25.781317 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c01f07008c2f7f9b6faadbc640e87b1.slice/crio-97182274c2cb415846e73a642e42cb93a702d869778ce1ca61b4e9089d0d5e76 WatchSource:0}: Error finding container 97182274c2cb415846e73a642e42cb93a702d869778ce1ca61b4e9089d0d5e76: Status 404 returned error can't find the container with id 97182274c2cb415846e73a642e42cb93a702d869778ce1ca61b4e9089d0d5e76 Jul 16 16:25:25.785686 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:25.785668 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:25:26.501960 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.501924 2560 apiserver.go:52] "Watching apiserver" Jul 16 16:25:26.509900 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.509870 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 16:25:26.511893 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.511862 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-kclc2","kube-system/global-pull-secret-syncer-nsqs2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal","openshift-multus/network-metrics-daemon-lqvdt","openshift-network-diagnostics/network-check-target-hzbgf","openshift-network-operator/iptables-alerter-m49m7","openshift-ovn-kubernetes/ovnkube-node-4wlc2","kube-system/konnectivity-agent-vfkkz","kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m","openshift-cluster-node-tuning-operator/tuned-s2fdg","openshift-dns/node-resolver-8h4f9","openshift-image-registry/node-ca-7zm9t","openshift-multus/multus-additional-cni-plugins-cvmkb"] Jul 16 16:25:26.514565 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.514544 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.516834 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.516810 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.519044 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.519023 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.521432 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.521410 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.521547 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.521482 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-768cx\"" Jul 16 16:25:26.521615 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.521410 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.521813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.521795 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 16:25:26.522560 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.522535 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:26.522668 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.522620 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:26.525151 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.525128 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:26.525233 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.525196 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:26.527475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.527456 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.529685 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.529648 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 16:25:26.530214 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.529767 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.530214 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.529888 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 16:25:26.530214 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.529904 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 16:25:26.530410 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.530226 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.531430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.531410 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.531507 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.531499 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 16:25:26.531563 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.531529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.532079 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532046 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.532266 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-log-socket\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-cni-bin\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-cni-netd\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-modprobe-d\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532383 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysctl-conf\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-run\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-system-cni-dir\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-kubelet\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-node-log\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovn-node-metrics-cert\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysconfig\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532528 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-kubernetes\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-sys\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532571 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7c8g\" (UniqueName: \"kubernetes.io/projected/b44deff2-378b-4023-8556-d2ab717305e4-kube-api-access-p7c8g\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532601 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cni-binary-copy\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-systemd-units\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-var-lib-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-ovn\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-run-netns\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-etc-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysctl-d\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-env-overrides\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovnkube-script-lib\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-systemd\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532854 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-var-lib-kubelet\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-host\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532884 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b44deff2-378b-4023-8556-d2ab717305e4-etc-tuned\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b44deff2-378b-4023-8556-d2ab717305e4-tmp\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532917 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-slash\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532940 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkmf8\" (UniqueName: \"kubernetes.io/projected/5d47c1f0-8e35-4c84-a715-573eaa81128f-kube-api-access-jkmf8\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.532975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-lib-modules\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532983 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.532999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-systemd\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533013 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-run-ovn-kubernetes\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovnkube-config\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533117 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cnibin\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-os-release\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533233 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533237 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdx8q\" (UniqueName: \"kubernetes.io/projected/b864d35e-dc48-4c12-a205-18fcd6fc6be9-kube-api-access-fdx8q\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533247 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533236 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 16:25:26.533525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.533270 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgbts\" (UniqueName: \"kubernetes.io/projected/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-kube-api-access-tgbts\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:26.534737 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.534719 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s2fd4\"" Jul 16 16:25:26.534897 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.534880 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.534973 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.534947 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:26.535841 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.535828 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tg6vv\"" Jul 16 16:25:26.535922 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.535899 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 16:25:26.536000 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.535965 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.536328 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.536313 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 16:25:26.536722 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.536707 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.538216 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.538199 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.538307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.538282 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xcll2\"" Jul 16 16:25:26.539166 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.539150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.541889 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.541872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gpjqx\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545379 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545515 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545516 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545554 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545567 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2zpt9\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545594 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545691 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nzn5f\"" Jul 16 16:25:26.545795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545702 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 16:25:26.546291 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.545937 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 16:25:26.546291 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.546075 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r4z2l\"" Jul 16 16:25:26.546291 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.546090 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 16:25:26.546291 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.546223 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 16:25:26.546438 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.546312 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 16:25:26.546438 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.546369 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bgmcf\"" Jul 16 16:25:26.546591 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.546573 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 16:25:26.548447 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.548432 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 16:25:26.555480 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.553049 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 16:25:26.618771 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.618737 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gc4zx" Jul 16 16:25:26.623246 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.623171 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 16:25:26.633634 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cnibin\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-device-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edee432b-41a0-4a35-924e-405d84cdb218-serviceca\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633757 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgbts\" (UniqueName: \"kubernetes.io/projected/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-kube-api-access-tgbts\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-cni-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.633803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-multus-certs\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633835 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gc4zx" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edee432b-41a0-4a35-924e-405d84cdb218-host\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-cni-netd\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633933 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-modprobe-d\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysctl-conf\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633975 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-run\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.633999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-system-cni-dir\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634083 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysconfig\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-sys\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/912d1490-4b6e-4acb-86ec-877cf5efbe4f-host-slash\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-systemd-units\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634178 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-var-lib-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634190 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-ovn\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634210 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-system-cni-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysctl-d\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-env-overrides\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovnkube-script-lib\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-var-lib-kubelet\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634344 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysconfig\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-host\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634384 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-modprobe-d\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b44deff2-378b-4023-8556-d2ab717305e4-etc-tuned\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634442 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-systemd-units\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634446 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cnibin\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b44deff2-378b-4023-8556-d2ab717305e4-tmp\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634483 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysctl-conf\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634491 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-socket-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/912d1490-4b6e-4acb-86ec-877cf5efbe4f-iptables-alerter-script\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634638 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-ovn\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.634817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634725 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634762 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-sysctl-d\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkmf8\" (UniqueName: \"kubernetes.io/projected/5d47c1f0-8e35-4c84-a715-573eaa81128f-kube-api-access-jkmf8\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-lib-modules\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634346 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-var-lib-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635194 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-netns\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635241 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-env-overrides\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635255 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-kubelet\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635281 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-var-lib-kubelet\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-conf-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635322 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-etc-kubernetes\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635337 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-host\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635345 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdzfx\" (UniqueName: \"kubernetes.io/projected/579023c8-e486-4414-98ab-42b1dbfc20c6-kube-api-access-qdzfx\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-lib-modules\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.636434 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-systemd\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovnkube-config\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.634391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-sys\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-os-release\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-socket-dir-parent\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635477 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-k8s-cni-cncf-io\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635548 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovnkube-script-lib\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrv5\" (UniqueName: \"kubernetes.io/projected/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-kube-api-access-rtrv5\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdx8q\" (UniqueName: \"kubernetes.io/projected/b864d35e-dc48-4c12-a205-18fcd6fc6be9-kube-api-access-fdx8q\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635618 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-cni-netd\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-registration-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635567 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-os-release\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.637357 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr7dw\" (UniqueName: \"kubernetes.io/projected/5562c64e-3b25-4911-93aa-760115c099ec-kube-api-access-tr7dw\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4d5h\" (UniqueName: \"kubernetes.io/projected/edee432b-41a0-4a35-924e-405d84cdb218-kube-api-access-k4d5h\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635701 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b864d35e-dc48-4c12-a205-18fcd6fc6be9-system-cni-dir\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-run\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ffb1bb7-d510-455e-828d-88887fa3c651-kubelet-config\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.635761 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-run-systemd\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.635832 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:27.135812037 +0000 UTC m=+2.943924029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-log-socket\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635881 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-cni-bin\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635900 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-log-socket\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635933 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cni-binary-copy\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.635957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-hostroot\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636019 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovnkube-config\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-cni-bin\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.638173 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-kubelet\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636348 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-node-log\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636387 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovn-node-metrics-cert\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636394 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-node-log\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-kubelet\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-kubernetes\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636517 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7c8g\" (UniqueName: \"kubernetes.io/projected/b44deff2-378b-4023-8556-d2ab717305e4-kube-api-access-p7c8g\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-cnibin\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-sys-fs\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-run-netns\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-etc-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636689 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b864d35e-dc48-4c12-a205-18fcd6fc6be9-cni-binary-copy\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-cni-bin\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-cni-multus\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/579023c8-e486-4414-98ab-42b1dbfc20c6-hosts-file\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-systemd\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.639005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636907 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-etc-openvswitch\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-daemon-config\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636966 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-kubernetes\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.636981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579023c8-e486-4414-98ab-42b1dbfc20c6-tmp-dir\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637009 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-run-netns\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/01259a01-eb1b-4e3f-b57a-ad02e9aad59d-agent-certs\") pod \"konnectivity-agent-vfkkz\" (UID: \"01259a01-eb1b-4e3f-b57a-ad02e9aad59d\") " pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637075 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b44deff2-378b-4023-8556-d2ab717305e4-etc-systemd\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-slash\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-os-release\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-slash\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637229 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-cni-binary-copy\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlq6b\" (UniqueName: \"kubernetes.io/projected/912d1490-4b6e-4acb-86ec-877cf5efbe4f-kube-api-access-dlq6b\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ffb1bb7-d510-455e-828d-88887fa3c651-dbus\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/01259a01-eb1b-4e3f-b57a-ad02e9aad59d-konnectivity-ca\") pod \"konnectivity-agent-vfkkz\" (UID: \"01259a01-eb1b-4e3f-b57a-ad02e9aad59d\") " pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637500 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-run-ovn-kubernetes\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.639857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.637541 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d47c1f0-8e35-4c84-a715-573eaa81128f-host-run-ovn-kubernetes\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.641001 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.640976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b44deff2-378b-4023-8556-d2ab717305e4-tmp\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.641337 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.641316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d47c1f0-8e35-4c84-a715-573eaa81128f-ovn-node-metrics-cert\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.643254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.642860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b44deff2-378b-4023-8556-d2ab717305e4-etc-tuned\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.646717 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.646657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkmf8\" (UniqueName: \"kubernetes.io/projected/5d47c1f0-8e35-4c84-a715-573eaa81128f-kube-api-access-jkmf8\") pod \"ovnkube-node-4wlc2\" (UID: \"5d47c1f0-8e35-4c84-a715-573eaa81128f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.649235 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.649186 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdx8q\" (UniqueName: \"kubernetes.io/projected/b864d35e-dc48-4c12-a205-18fcd6fc6be9-kube-api-access-fdx8q\") pod \"multus-additional-cni-plugins-cvmkb\" (UID: \"b864d35e-dc48-4c12-a205-18fcd6fc6be9\") " pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.652015 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.651761 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7c8g\" (UniqueName: \"kubernetes.io/projected/b44deff2-378b-4023-8556-d2ab717305e4-kube-api-access-p7c8g\") pod \"tuned-s2fdg\" (UID: \"b44deff2-378b-4023-8556-d2ab717305e4\") " pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.654051 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.654003 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgbts\" (UniqueName: \"kubernetes.io/projected/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-kube-api-access-tgbts\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:26.656219 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.656173 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"0aa522b1d98aef62f303d8039fdc50f5","Type":"ContainerStarted","Data":"9d3f64079066706c9d7b24f2f4fb347f3a5b187f0b58c9c28e6d73f578c83972"} Jul 16 16:25:26.657996 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.657974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" event={"ID":"3c01f07008c2f7f9b6faadbc640e87b1","Type":"ContainerStarted","Data":"97182274c2cb415846e73a642e42cb93a702d869778ce1ca61b4e9089d0d5e76"} Jul 16 16:25:26.738367 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/912d1490-4b6e-4acb-86ec-877cf5efbe4f-host-slash\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-system-cni-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-socket-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/912d1490-4b6e-4acb-86ec-877cf5efbe4f-iptables-alerter-script\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-netns\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-kubelet\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-conf-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-etc-kubernetes\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdzfx\" (UniqueName: \"kubernetes.io/projected/579023c8-e486-4414-98ab-42b1dbfc20c6-kube-api-access-qdzfx\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-socket-dir-parent\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-k8s-cni-cncf-io\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrv5\" (UniqueName: \"kubernetes.io/projected/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-kube-api-access-rtrv5\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-registration-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr7dw\" (UniqueName: \"kubernetes.io/projected/5562c64e-3b25-4911-93aa-760115c099ec-kube-api-access-tr7dw\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4d5h\" (UniqueName: \"kubernetes.io/projected/edee432b-41a0-4a35-924e-405d84cdb218-kube-api-access-k4d5h\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ffb1bb7-d510-455e-828d-88887fa3c651-kubelet-config\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.738872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738894 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-hostroot\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-cnibin\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-sys-fs\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.738993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-cni-bin\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-cni-multus\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/579023c8-e486-4414-98ab-42b1dbfc20c6-hosts-file\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739046 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-daemon-config\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579023c8-e486-4414-98ab-42b1dbfc20c6-tmp-dir\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/01259a01-eb1b-4e3f-b57a-ad02e9aad59d-agent-certs\") pod \"konnectivity-agent-vfkkz\" (UID: \"01259a01-eb1b-4e3f-b57a-ad02e9aad59d\") " pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-os-release\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739139 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-cni-binary-copy\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlq6b\" (UniqueName: \"kubernetes.io/projected/912d1490-4b6e-4acb-86ec-877cf5efbe4f-kube-api-access-dlq6b\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ffb1bb7-d510-455e-828d-88887fa3c651-dbus\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/01259a01-eb1b-4e3f-b57a-ad02e9aad59d-konnectivity-ca\") pod \"konnectivity-agent-vfkkz\" (UID: \"01259a01-eb1b-4e3f-b57a-ad02e9aad59d\") " pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-device-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.739277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edee432b-41a0-4a35-924e-405d84cdb218-serviceca\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-cni-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-multus-certs\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edee432b-41a0-4a35-924e-405d84cdb218-host\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739534 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edee432b-41a0-4a35-924e-405d84cdb218-host\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/912d1490-4b6e-4acb-86ec-877cf5efbe4f-host-slash\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739767 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-system-cni-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.739899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.739890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-socket-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.740461 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/912d1490-4b6e-4acb-86ec-877cf5efbe4f-iptables-alerter-script\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.740518 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740508 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-netns\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.740564 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-kubelet\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.740609 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740587 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-conf-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.740661 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740634 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-etc-kubernetes\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.740938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740919 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-socket-dir-parent\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.741018 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.740981 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-k8s-cni-cncf-io\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.741309 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.741286 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-registration-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.750507 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579023c8-e486-4414-98ab-42b1dbfc20c6-tmp-dir\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.750967 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ffb1bb7-d510-455e-828d-88887fa3c651-kubelet-config\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.751125 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.751261 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:27.251241004 +0000 UTC m=+3.059352999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751510 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-hostroot\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751618 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751698 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-cnibin\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751758 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-sys-fs\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-cni-bin\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751884 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-var-lib-cni-multus\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.751947 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/579023c8-e486-4414-98ab-42b1dbfc20c6-hosts-file\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.752572 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-daemon-config\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.753122 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-os-release\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.753991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ffb1bb7-d510-455e-828d-88887fa3c651-dbus\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:26.754143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.754088 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-cni-binary-copy\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754982 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.754179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5562c64e-3b25-4911-93aa-760115c099ec-device-dir\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.754982 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.754608 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/01259a01-eb1b-4e3f-b57a-ad02e9aad59d-konnectivity-ca\") pod \"konnectivity-agent-vfkkz\" (UID: \"01259a01-eb1b-4e3f-b57a-ad02e9aad59d\") " pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.754982 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.754620 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edee432b-41a0-4a35-924e-405d84cdb218-serviceca\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.754982 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.754694 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-host-run-multus-certs\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.754982 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.754754 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-multus-cni-dir\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.757898 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.757848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr7dw\" (UniqueName: \"kubernetes.io/projected/5562c64e-3b25-4911-93aa-760115c099ec-kube-api-access-tr7dw\") pod \"aws-ebs-csi-driver-node-x6g7m\" (UID: \"5562c64e-3b25-4911-93aa-760115c099ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.759102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.759025 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdzfx\" (UniqueName: \"kubernetes.io/projected/579023c8-e486-4414-98ab-42b1dbfc20c6-kube-api-access-qdzfx\") pod \"node-resolver-8h4f9\" (UID: \"579023c8-e486-4414-98ab-42b1dbfc20c6\") " pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.759248 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.759230 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:26.759304 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.759256 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:26.759304 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.759271 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:26.759407 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:26.759337 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:27.259319813 +0000 UTC m=+3.067431823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:26.761343 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.761298 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/01259a01-eb1b-4e3f-b57a-ad02e9aad59d-agent-certs\") pod \"konnectivity-agent-vfkkz\" (UID: \"01259a01-eb1b-4e3f-b57a-ad02e9aad59d\") " pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.762872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.762829 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrv5\" (UniqueName: \"kubernetes.io/projected/e9d7f57d-bf2a-4015-bfc3-1d68c85e188e-kube-api-access-rtrv5\") pod \"multus-kclc2\" (UID: \"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e\") " pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.769859 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.769809 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4d5h\" (UniqueName: \"kubernetes.io/projected/edee432b-41a0-4a35-924e-405d84cdb218-kube-api-access-k4d5h\") pod \"node-ca-7zm9t\" (UID: \"edee432b-41a0-4a35-924e-405d84cdb218\") " pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.772929 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.772882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlq6b\" (UniqueName: \"kubernetes.io/projected/912d1490-4b6e-4acb-86ec-877cf5efbe4f-kube-api-access-dlq6b\") pod \"iptables-alerter-m49m7\" (UID: \"912d1490-4b6e-4acb-86ec-877cf5efbe4f\") " pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.825995 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.825964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:26.835231 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.835201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" Jul 16 16:25:26.841497 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.841472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" Jul 16 16:25:26.847284 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.847259 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m49m7" Jul 16 16:25:26.855661 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.855472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8h4f9" Jul 16 16:25:26.861948 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.861784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kclc2" Jul 16 16:25:26.867084 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.867044 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" Jul 16 16:25:26.872485 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.872468 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7zm9t" Jul 16 16:25:26.877392 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:26.877370 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:26.993913 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:26.993851 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedee432b_41a0_4a35_924e_405d84cdb218.slice/crio-8ac08e352c1a177ea02c8984ed1a48fd4a02e4923153f16321e4edb6123d184f WatchSource:0}: Error finding container 8ac08e352c1a177ea02c8984ed1a48fd4a02e4923153f16321e4edb6123d184f: Status 404 returned error can't find the container with id 8ac08e352c1a177ea02c8984ed1a48fd4a02e4923153f16321e4edb6123d184f Jul 16 16:25:26.996461 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:26.996437 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912d1490_4b6e_4acb_86ec_877cf5efbe4f.slice/crio-e6c5486c9bfdc8c912f49c14e31dec3f41002175ab9a8e765c031b87587ecb95 WatchSource:0}: Error finding container e6c5486c9bfdc8c912f49c14e31dec3f41002175ab9a8e765c031b87587ecb95: Status 404 returned error can't find the container with id e6c5486c9bfdc8c912f49c14e31dec3f41002175ab9a8e765c031b87587ecb95 Jul 16 16:25:27.034083 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.023289 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44deff2_378b_4023_8556_d2ab717305e4.slice/crio-721c7973a95146bec455a421b0e45a6b838cf888b06dfe8c508af580b5c2169e WatchSource:0}: Error finding container 721c7973a95146bec455a421b0e45a6b838cf888b06dfe8c508af580b5c2169e: Status 404 returned error can't find the container with id 721c7973a95146bec455a421b0e45a6b838cf888b06dfe8c508af580b5c2169e Jul 16 16:25:27.034083 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.024863 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d47c1f0_8e35_4c84_a715_573eaa81128f.slice/crio-1a7f1a95127429087455b1a702ff1cc7f0955f217aa58358545e54de4b3a3034 WatchSource:0}: Error finding container 1a7f1a95127429087455b1a702ff1cc7f0955f217aa58358545e54de4b3a3034: Status 404 returned error can't find the container with id 1a7f1a95127429087455b1a702ff1cc7f0955f217aa58358545e54de4b3a3034 Jul 16 16:25:27.034083 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.025857 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb864d35e_dc48_4c12_a205_18fcd6fc6be9.slice/crio-b154a78468112f1fbe8678e64a116efc643252d6a3985f462c705148d00464b3 WatchSource:0}: Error finding container b154a78468112f1fbe8678e64a116efc643252d6a3985f462c705148d00464b3: Status 404 returned error can't find the container with id b154a78468112f1fbe8678e64a116efc643252d6a3985f462c705148d00464b3 Jul 16 16:25:27.034083 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.026751 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5562c64e_3b25_4911_93aa_760115c099ec.slice/crio-4394d348634e9852527b0667099da03bcce9bc41c58454f511d32f6a6eb5a806 WatchSource:0}: Error finding container 4394d348634e9852527b0667099da03bcce9bc41c58454f511d32f6a6eb5a806: Status 404 returned error can't find the container with id 4394d348634e9852527b0667099da03bcce9bc41c58454f511d32f6a6eb5a806 Jul 16 16:25:27.034083 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.028723 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579023c8_e486_4414_98ab_42b1dbfc20c6.slice/crio-5317b6352a73b74af213605c337bd382b197996366dd0cc5ae02ccfdfceed7f7 WatchSource:0}: Error finding container 5317b6352a73b74af213605c337bd382b197996366dd0cc5ae02ccfdfceed7f7: Status 404 returned error can't find the container with id 5317b6352a73b74af213605c337bd382b197996366dd0cc5ae02ccfdfceed7f7 Jul 16 16:25:27.034083 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.029371 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d7f57d_bf2a_4015_bfc3_1d68c85e188e.slice/crio-9e25fecf33859b0a3c45981c66df51a8afce725c4b0a81ad26063c893ebd7c43 WatchSource:0}: Error finding container 9e25fecf33859b0a3c45981c66df51a8afce725c4b0a81ad26063c893ebd7c43: Status 404 returned error can't find the container with id 9e25fecf33859b0a3c45981c66df51a8afce725c4b0a81ad26063c893ebd7c43 Jul 16 16:25:27.034791 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:25:27.034759 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01259a01_eb1b_4e3f_b57a_ad02e9aad59d.slice/crio-7f2f476915c652cf6281d140a4d106183310b37e18e1124c5b377e0f6a2baca2 WatchSource:0}: Error finding container 7f2f476915c652cf6281d140a4d106183310b37e18e1124c5b377e0f6a2baca2: Status 404 returned error can't find the container with id 7f2f476915c652cf6281d140a4d106183310b37e18e1124c5b377e0f6a2baca2 Jul 16 16:25:27.143121 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.143083 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:27.143265 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.143243 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:27.143358 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.143334 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:28.143313588 +0000 UTC m=+3.951425584 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:27.345857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.345611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:27.346018 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.345880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:27.346018 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.345797 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:27.346018 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.345979 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:27.346018 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.345988 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:27.346018 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.345995 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:27.346241 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.346044 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:28.34602914 +0000 UTC m=+4.154141132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:27.346241 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:27.346075 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:28.346051438 +0000 UTC m=+4.154163430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:27.639195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.639149 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 16:20:26 +0000 UTC" deadline="2027-12-27 14:51:48.861266345 +0000 UTC" Jul 16 16:25:27.639195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.639194 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12694h26m21.222076623s" Jul 16 16:25:27.662013 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.661856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kclc2" event={"ID":"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e","Type":"ContainerStarted","Data":"9e25fecf33859b0a3c45981c66df51a8afce725c4b0a81ad26063c893ebd7c43"} Jul 16 16:25:27.663449 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.663418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7zm9t" event={"ID":"edee432b-41a0-4a35-924e-405d84cdb218","Type":"ContainerStarted","Data":"8ac08e352c1a177ea02c8984ed1a48fd4a02e4923153f16321e4edb6123d184f"} Jul 16 16:25:27.665548 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.665480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8h4f9" event={"ID":"579023c8-e486-4414-98ab-42b1dbfc20c6","Type":"ContainerStarted","Data":"5317b6352a73b74af213605c337bd382b197996366dd0cc5ae02ccfdfceed7f7"} Jul 16 16:25:27.667802 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.667655 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" event={"ID":"5562c64e-3b25-4911-93aa-760115c099ec","Type":"ContainerStarted","Data":"4394d348634e9852527b0667099da03bcce9bc41c58454f511d32f6a6eb5a806"} Jul 16 16:25:27.669435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.669307 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerStarted","Data":"b154a78468112f1fbe8678e64a116efc643252d6a3985f462c705148d00464b3"} Jul 16 16:25:27.670956 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.670840 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vfkkz" event={"ID":"01259a01-eb1b-4e3f-b57a-ad02e9aad59d","Type":"ContainerStarted","Data":"7f2f476915c652cf6281d140a4d106183310b37e18e1124c5b377e0f6a2baca2"} Jul 16 16:25:27.673303 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.673276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"1a7f1a95127429087455b1a702ff1cc7f0955f217aa58358545e54de4b3a3034"} Jul 16 16:25:27.674851 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.674824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" event={"ID":"b44deff2-378b-4023-8556-d2ab717305e4","Type":"ContainerStarted","Data":"721c7973a95146bec455a421b0e45a6b838cf888b06dfe8c508af580b5c2169e"} Jul 16 16:25:27.681468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.681430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m49m7" event={"ID":"912d1490-4b6e-4acb-86ec-877cf5efbe4f","Type":"ContainerStarted","Data":"e6c5486c9bfdc8c912f49c14e31dec3f41002175ab9a8e765c031b87587ecb95"} Jul 16 16:25:27.683867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.683840 2560 generic.go:356] "Generic (PLEG): container finished" podID="0aa522b1d98aef62f303d8039fdc50f5" containerID="89ab2eb14d8117d4e1fba77c87ec5206f9a1db1b29a4b5ebdd9c4b8c36534963" exitCode=0 Jul 16 16:25:27.683953 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:27.683879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"0aa522b1d98aef62f303d8039fdc50f5","Type":"ContainerDied","Data":"89ab2eb14d8117d4e1fba77c87ec5206f9a1db1b29a4b5ebdd9c4b8c36534963"} Jul 16 16:25:28.151590 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.151547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:28.151744 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.151726 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:28.151821 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.151794 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:30.151773578 +0000 UTC m=+5.959885574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:28.353768 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.353684 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:28.353768 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.353735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:28.353984 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.353915 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:28.353984 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.353982 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:30.35396044 +0000 UTC m=+6.162072440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:28.354512 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.354493 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:28.354596 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.354519 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:28.354596 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.354532 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:28.354596 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.354580 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:30.354564233 +0000 UTC m=+6.162676228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:28.642968 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.639476 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 16:20:26 +0000 UTC" deadline="2028-03-05 15:17:33.489830385 +0000 UTC" Jul 16 16:25:28.642968 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.639533 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14350h52m4.850301463s" Jul 16 16:25:28.652506 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.652464 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:28.652665 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.652618 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:28.652731 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.652698 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:28.652867 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.652817 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:28.652953 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.652894 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:28.653100 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:28.652998 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:28.703633 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.703432 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"0aa522b1d98aef62f303d8039fdc50f5","Type":"ContainerStarted","Data":"5cbcc386296a6c17b2b3a73e140b55cbb6446726681ac6e3d5c57462ba39f985"} Jul 16 16:25:28.720419 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.720388 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" event={"ID":"3c01f07008c2f7f9b6faadbc640e87b1","Type":"ContainerStarted","Data":"520a9e59c14636b6a517d9a0b14896f09dff8c81db959f4e12651fbe7dd3c4a5"} Jul 16 16:25:28.766884 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.766821 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" podStartSLOduration=3.766796355 podStartE2EDuration="3.766796355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:28.734933115 +0000 UTC m=+4.543045131" watchObservedRunningTime="2026-07-16 16:25:28.766796355 +0000 UTC m=+4.574908371" Jul 16 16:25:28.767764 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:28.767725 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" podStartSLOduration=3.7677117449999997 podStartE2EDuration="3.767711745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:28.767245135 +0000 UTC m=+4.575357155" watchObservedRunningTime="2026-07-16 16:25:28.767711745 +0000 UTC m=+4.575823760" Jul 16 16:25:30.169467 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:30.169428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:30.169936 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.169601 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:30.169936 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.169661 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:34.169644051 +0000 UTC m=+9.977756046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:30.370722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:30.370774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.370908 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.370971 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:34.37095289 +0000 UTC m=+10.179064905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.371206 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.371230 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.371242 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:30.371353 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.371288 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:34.371272291 +0000 UTC m=+10.179384298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:30.650432 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:30.649885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:30.650432 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:30.649910 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:30.650432 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.650018 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:30.650432 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:30.650104 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:30.650432 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.650215 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:30.650432 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:30.650281 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:32.653020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:32.652240 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:32.653020 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:32.652368 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:32.653020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:32.652747 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:32.653020 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:32.652839 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:32.653020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:32.652889 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:32.653020 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:32.652979 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:34.206403 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:34.205911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:34.206403 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.206030 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:34.206403 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.206101 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:42.206082034 +0000 UTC m=+18.014194075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:34.407468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:34.407432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:34.407468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:34.407470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:34.407705 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.407579 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:34.407705 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.407630 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:42.407616716 +0000 UTC m=+18.215728709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:34.407992 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.407954 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:34.407992 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.407992 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:34.408166 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.408006 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:34.408166 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.408084 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:42.408043968 +0000 UTC m=+18.216155960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:34.650778 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:34.650691 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:34.650934 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.650792 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:34.650934 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:34.650804 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:34.650934 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:34.650840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:34.650934 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.650898 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:34.651115 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:34.651009 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:36.649377 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:36.649346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:36.649796 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:36.649464 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:36.649856 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:36.649845 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:36.649970 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:36.649941 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:36.650095 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:36.650004 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:36.650095 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:36.650076 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:38.649419 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:38.649385 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:38.649804 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:38.649393 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:38.649804 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:38.649508 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:38.649804 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:38.649594 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:38.649804 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:38.649395 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:38.649804 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:38.649679 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:40.650163 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:40.650117 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:40.650657 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:40.650129 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:40.650657 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:40.650245 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:40.650657 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:40.650258 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:40.650657 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:40.650376 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:40.650657 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:40.650473 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:42.268545 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:42.268505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:42.269039 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.268650 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:42.269039 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.268717 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:58.268698329 +0000 UTC m=+34.076810327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:42.470461 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:42.470420 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:42.470638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:42.470473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:42.470638 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.470590 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:42.470638 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.470614 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:42.470638 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.470641 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:42.470829 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.470655 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:42.470829 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.470657 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:58.470638515 +0000 UTC m=+34.278750512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:42.470829 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.470703 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:25:58.470686968 +0000 UTC m=+34.278798961 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:42.650366 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:42.650294 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:42.650366 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:42.650359 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:42.650578 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:42.650368 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:42.650578 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.650454 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:42.650578 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.650527 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:42.650720 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:42.650616 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:44.650019 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:44.649981 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:44.650430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:44.650156 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:44.650430 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:44.650160 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:44.650430 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:44.650284 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:44.650430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:44.650349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:44.650430 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:44.650416 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:46.650113 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:46.650077 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:46.650678 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:46.650120 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:46.650678 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:46.650204 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:46.650678 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:46.650292 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:46.650678 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:46.650341 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:46.650678 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:46.650441 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:47.767937 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.767497 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"605724035e1bab0a3674d79f128439a05ea2babc2b06d844542ac5c8d2746c6a"} Jul 16 16:25:47.768938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.767983 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"555e283f3f1e87029e45f74ea5f1056fb35ecc85b8fec27565ba23aedcacab1d"} Jul 16 16:25:47.768938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.768033 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"026febd6478094fbbb8efab60f3506c8a5e945b65eb621a0fad78d586e068214"} Jul 16 16:25:47.768938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.768095 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"51affe673d2f3214e5acc7076daef34779c7c7edf23b88f9247a93b801484d26"} Jul 16 16:25:47.768938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.768124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"926c8a4beeca5025f0c7405172ada72d7189845e2c8f69295c76c61e9422808f"} Jul 16 16:25:47.768938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.768172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"c5f542728cad9f12d75d6790e5d9b27c2bc6a76bea6573f6c3aa9fcbb86d71c1"} Jul 16 16:25:47.770889 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.769385 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" event={"ID":"b44deff2-378b-4023-8556-d2ab717305e4","Type":"ContainerStarted","Data":"5a84f67ab42e74c8bd208e748d4444dc99d11178ed6da89ebc9bd18833b03286"} Jul 16 16:25:47.771805 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.771781 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kclc2" event={"ID":"e9d7f57d-bf2a-4015-bfc3-1d68c85e188e","Type":"ContainerStarted","Data":"1c4e0a6cb606de8f03c4cd3adfe306388413aef45df45dea5e8473d21c357988"} Jul 16 16:25:47.773318 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.773293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7zm9t" event={"ID":"edee432b-41a0-4a35-924e-405d84cdb218","Type":"ContainerStarted","Data":"dc4963de741a2410f574406abe658cb085a0ff3e3d468d73b86bf9685a91c496"} Jul 16 16:25:47.774851 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.774831 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8h4f9" event={"ID":"579023c8-e486-4414-98ab-42b1dbfc20c6","Type":"ContainerStarted","Data":"1b89c82dae23cb5dd52d16be766ac08b97b4c2f1763d878296bc9c89dcc2efbc"} Jul 16 16:25:47.776452 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.776431 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" event={"ID":"5562c64e-3b25-4911-93aa-760115c099ec","Type":"ContainerStarted","Data":"d7d321a58473fc46bb32c763aa7ff70bcf8d178f78904db6c6cc29ac31803a2c"} Jul 16 16:25:47.777935 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.777914 2560 generic.go:356] "Generic (PLEG): container finished" podID="b864d35e-dc48-4c12-a205-18fcd6fc6be9" containerID="a96c6d7c17a27a0a5fc725423593c9943657e4269612f65ef3c8e0a2aaf3f42a" exitCode=0 Jul 16 16:25:47.778035 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.777970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerDied","Data":"a96c6d7c17a27a0a5fc725423593c9943657e4269612f65ef3c8e0a2aaf3f42a"} Jul 16 16:25:47.779789 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.779771 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vfkkz" event={"ID":"01259a01-eb1b-4e3f-b57a-ad02e9aad59d","Type":"ContainerStarted","Data":"50b2d3d18c6ed0bd3af20dc2cc02ec6ed382deb6f282c3dacdc0c033c9bb1710"} Jul 16 16:25:47.794815 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.794773 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s2fdg" podStartSLOduration=3.972858387 podStartE2EDuration="23.794761035s" totalImagesPullingTime="19.821902648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:47.794446974 +0000 UTC m=+23.602558988" watchObservedRunningTime="2026-07-16 16:25:47.794761035 +0000 UTC m=+23.602873049" Jul 16 16:25:47.815508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.815455 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8h4f9" podStartSLOduration=4.000576791 podStartE2EDuration="23.815436473s" totalImagesPullingTime="19.814859682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:47.814861509 +0000 UTC m=+23.622973523" watchObservedRunningTime="2026-07-16 16:25:47.815436473 +0000 UTC m=+23.623548491" Jul 16 16:25:47.844648 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.844589 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7zm9t" podStartSLOduration=3.017098192 podStartE2EDuration="22.84457057s" totalImagesPullingTime="19.827472378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:47.843697213 +0000 UTC m=+23.651809260" watchObservedRunningTime="2026-07-16 16:25:47.84457057 +0000 UTC m=+23.652682584" Jul 16 16:25:47.886856 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:47.886804 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vfkkz" podStartSLOduration=3.099460637 podStartE2EDuration="22.886787932s" totalImagesPullingTime="19.787327295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:47.861267621 +0000 UTC m=+23.669379654" watchObservedRunningTime="2026-07-16 16:25:47.886787932 +0000 UTC m=+23.694899945" Jul 16 16:25:48.188559 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.188533 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 16:25:48.601739 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.601579 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T16:25:48.188555734Z","UUID":"d289a9bf-38e6-4733-bd51-6597c26fca42","Handler":null,"Name":"","Endpoint":""} Jul 16 16:25:48.603125 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.603105 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 16:25:48.603226 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.603141 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 16:25:48.650171 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.650140 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:48.650171 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.650169 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:48.650382 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.650170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:48.650382 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:48.650238 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:48.650382 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:48.650353 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:48.650498 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:48.650413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:48.783867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.783825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" event={"ID":"5562c64e-3b25-4911-93aa-760115c099ec","Type":"ContainerStarted","Data":"4e3843bf6a4a493f25f2682af82b2a14a481254d3a02e96e6cd4b4e6de87590e"} Jul 16 16:25:48.785688 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.785651 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m49m7" event={"ID":"912d1490-4b6e-4acb-86ec-877cf5efbe4f","Type":"ContainerStarted","Data":"3cf5d1d1b0abb0d20d84575dbb46c8b5f4ed820f6dfa7250555df50bb75c78ce"} Jul 16 16:25:48.801516 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.801466 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m49m7" podStartSLOduration=4.974045902 podStartE2EDuration="24.801454731s" totalImagesPullingTime="19.827408829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:48.801325623 +0000 UTC m=+24.609437640" watchObservedRunningTime="2026-07-16 16:25:48.801454731 +0000 UTC m=+24.609566745" Jul 16 16:25:48.802028 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:48.801994 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kclc2" podStartSLOduration=4.876164055 podStartE2EDuration="24.801982901s" totalImagesPullingTime="19.925818846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:47.937199683 +0000 UTC m=+23.745311688" watchObservedRunningTime="2026-07-16 16:25:48.801982901 +0000 UTC m=+24.610094917" Jul 16 16:25:49.362938 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.362914 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:49.363442 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.363423 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:49.791087 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.790510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" event={"ID":"5562c64e-3b25-4911-93aa-760115c099ec","Type":"ContainerStarted","Data":"9e8eafd296d2a4aafe49c9822a616c00ef4fa0e5b5a93df292e997a5145e1762"} Jul 16 16:25:49.795329 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.795293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"7e95f47ff148d43d11a6dbbb99d8e2ea6c2be9211cfcf1dc4d4b13612f4c2cc1"} Jul 16 16:25:49.795681 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.795660 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:49.796191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.796173 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vfkkz" Jul 16 16:25:49.817629 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:49.817580 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6g7m" podStartSLOduration=2.88602854 podStartE2EDuration="24.81756434s" totalImagesPullingTime="21.9315358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:49.817322343 +0000 UTC m=+25.625434356" watchObservedRunningTime="2026-07-16 16:25:49.81756434 +0000 UTC m=+25.625676367" Jul 16 16:25:50.650211 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:50.650174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:50.650465 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:50.650174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:50.650465 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:50.650309 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:50.650465 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:50.650186 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:50.650465 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:50.650391 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:50.650602 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:50.650473 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:52.652190 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:52.651990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:52.652663 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:52.652288 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:52.652733 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:52.652682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:52.652808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:52.652785 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:52.652869 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:52.652840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:52.652923 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:52.652910 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:53.807099 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.806836 2560 generic.go:356] "Generic (PLEG): container finished" podID="b864d35e-dc48-4c12-a205-18fcd6fc6be9" containerID="283151f73513c6fb5ba13bb60062c47a08cf5060d0069cf598012826d3e65716" exitCode=0 Jul 16 16:25:53.807099 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.806883 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerDied","Data":"283151f73513c6fb5ba13bb60062c47a08cf5060d0069cf598012826d3e65716"} Jul 16 16:25:53.810738 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.810714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" event={"ID":"5d47c1f0-8e35-4c84-a715-573eaa81128f","Type":"ContainerStarted","Data":"f7b36dc323aa879bb8950b4042940accfb7f558b1fd67b5ed9459549c37aeecd"} Jul 16 16:25:53.810983 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.810965 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:53.811090 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.811071 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:53.811145 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.811121 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:53.826694 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.826672 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:53.826806 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.826774 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:25:53.866545 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:53.866496 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" podStartSLOduration=10.00337242 podStartE2EDuration="29.866479487s" totalImagesPullingTime="19.863107067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:25:53.865983038 +0000 UTC m=+29.674095052" watchObservedRunningTime="2026-07-16 16:25:53.866479487 +0000 UTC m=+29.674591500" Jul 16 16:25:54.650390 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:54.650356 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:54.650651 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:54.650434 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:54.650651 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:54.650525 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:54.650651 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:54.650608 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:54.650651 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:54.650651 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:54.650785 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:54.650730 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:55.693580 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.693544 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nsqs2"] Jul 16 16:25:55.694091 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.693644 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:55.694091 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:55.693736 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:55.696600 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.696543 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzbgf"] Jul 16 16:25:55.696739 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.696651 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:55.696782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:55.696731 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:55.697014 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.696997 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqvdt"] Jul 16 16:25:55.697096 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.697089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:55.697181 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:55.697164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:55.819200 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.819167 2560 generic.go:356] "Generic (PLEG): container finished" podID="b864d35e-dc48-4c12-a205-18fcd6fc6be9" containerID="a5d33d0cbc1ef6e35c02fb3e4ad3799bea53ebf29632cd8f51acfddca820b76d" exitCode=0 Jul 16 16:25:55.819297 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:55.819275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerDied","Data":"a5d33d0cbc1ef6e35c02fb3e4ad3799bea53ebf29632cd8f51acfddca820b76d"} Jul 16 16:25:56.824530 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:56.824501 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerStarted","Data":"3de658e0e6e285376216d066d5f2cf70b6f2bf3b4e99825187fee2efb9f38218"} Jul 16 16:25:57.649736 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:57.649661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:57.649736 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:57.649688 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:57.649736 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:57.649696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:57.649922 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:57.649782 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqvdt" podUID="e86a25b1-e8e9-4b5d-b2c6-4011245d5870" Jul 16 16:25:57.649922 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:57.649824 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzbgf" podUID="4406ffec-a202-45c2-98bb-c83cc037e173" Jul 16 16:25:57.649922 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:57.649898 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsqs2" podUID="1ffb1bb7-d510-455e-828d-88887fa3c651" Jul 16 16:25:57.829338 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:57.829307 2560 generic.go:356] "Generic (PLEG): container finished" podID="b864d35e-dc48-4c12-a205-18fcd6fc6be9" containerID="3de658e0e6e285376216d066d5f2cf70b6f2bf3b4e99825187fee2efb9f38218" exitCode=0 Jul 16 16:25:57.829697 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:57.829362 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerDied","Data":"3de658e0e6e285376216d066d5f2cf70b6f2bf3b4e99825187fee2efb9f38218"} Jul 16 16:25:58.286902 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:58.286871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:58.287083 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.287015 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:58.287138 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.287095 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs podName:e86a25b1-e8e9-4b5d-b2c6-4011245d5870 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:30.287075436 +0000 UTC m=+66.095187564 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs") pod "network-metrics-daemon-lqvdt" (UID: "e86a25b1-e8e9-4b5d-b2c6-4011245d5870") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 16:25:58.487866 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:58.487817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:58.488046 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:58.487876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:58.488046 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.487996 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 16:25:58.488046 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.488019 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 16:25:58.488046 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.488016 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:58.488046 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.488029 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4bnvv for pod openshift-network-diagnostics/network-check-target-hzbgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:58.488249 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.488108 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv podName:4406ffec-a202-45c2-98bb-c83cc037e173 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:30.488093701 +0000 UTC m=+66.296205694 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnvv" (UniqueName: "kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv") pod "network-check-target-hzbgf" (UID: "4406ffec-a202-45c2-98bb-c83cc037e173") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 16:25:58.488249 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:58.488125 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret podName:1ffb1bb7-d510-455e-828d-88887fa3c651 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:30.488118476 +0000 UTC m=+66.296230468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret") pod "global-pull-secret-syncer-nsqs2" (UID: "1ffb1bb7-d510-455e-828d-88887fa3c651") : object "kube-system"/"original-pull-secret" not registered Jul 16 16:25:59.508830 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.508799 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeReady" Jul 16 16:25:59.509260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.508948 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 16:25:59.566864 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.566828 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b4cdb78f7-jz5rc"] Jul 16 16:25:59.569487 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.569461 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dlqfr"] Jul 16 16:25:59.569621 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.569584 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.572617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.572593 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd"] Jul 16 16:25:59.572755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.572719 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.574783 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.574759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.576968 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.576945 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.577127 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.576969 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 16:25:59.577664 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.577633 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-v7cw2\"" Jul 16 16:25:59.578462 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.578425 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 16:25:59.578462 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.578450 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 16:25:59.578635 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.578491 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 16:25:59.578635 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.578431 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.579104 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.578867 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 16:25:59.580583 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.580561 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpvsj\"" Jul 16 16:25:59.580735 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.580645 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.580917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.580860 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lw7bt\"" Jul 16 16:25:59.581124 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.581088 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 16:25:59.581862 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.581839 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 16:25:59.582416 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.582392 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.590146 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.590126 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 16:25:59.596897 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.596262 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dlqfr"] Jul 16 16:25:59.607199 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.607180 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 16:25:59.607400 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.607385 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b4cdb78f7-jz5rc"] Jul 16 16:25:59.608404 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.608365 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42"] Jul 16 16:25:59.613486 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.613459 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b"] Jul 16 16:25:59.614278 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.614230 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.617085 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.616948 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 16:25:59.617170 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.617017 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 16:25:59.617360 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.617342 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tjjpr"] Jul 16 16:25:59.617458 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.617384 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6vfzc\"" Jul 16 16:25:59.617703 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.617684 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" Jul 16 16:25:59.618413 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.618374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.618567 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.618555 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.620255 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.620235 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df"] Jul 16 16:25:59.620463 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.620446 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:25:59.624082 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.623890 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 16:25:59.624169 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.624138 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 16:25:59.624169 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.624153 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.624380 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.624361 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rxjzg\"" Jul 16 16:25:59.624518 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.624408 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.625531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.625516 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7fb97c79fd-4cvvx"] Jul 16 16:25:59.625659 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.625635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" Jul 16 16:25:59.625836 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.625819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fw2sx\"" Jul 16 16:25:59.628549 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.628530 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-6g8qg"] Jul 16 16:25:59.628703 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.628689 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.632383 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.632363 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z"] Jul 16 16:25:59.632537 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.632522 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.634624 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.634607 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.634838 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.634640 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.635591 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.635572 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k"] Jul 16 16:25:59.635810 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.635790 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:25:59.639157 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.639137 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42"] Jul 16 16:25:59.639252 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.639172 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd"] Jul 16 16:25:59.639252 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.639186 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df"] Jul 16 16:25:59.639334 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.639254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.640825 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.640806 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-47hc6\"" Jul 16 16:25:59.641077 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641048 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 16:25:59.641220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641205 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-frgb4\"" Jul 16 16:25:59.641390 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641376 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 16:25:59.641533 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641514 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 16:25:59.641791 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641773 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 16:25:59.641858 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641816 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.641920 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.641776 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.642082 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642042 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.642331 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642312 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vplzt\"" Jul 16 16:25:59.642447 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642430 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.642501 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642490 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 16:25:59.642552 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642531 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.642603 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642592 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nh98m\"" Jul 16 16:25:59.642825 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642809 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 16:25:59.642932 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.642842 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.643079 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.643042 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 16:25:59.643167 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.643141 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.643228 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.643048 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 16:25:59.648595 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.647501 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hcgqh\"" Jul 16 16:25:59.648595 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.647695 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.648595 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.647843 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 16:25:59.648595 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.648190 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 16:25:59.649024 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.648697 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tjjpr"] Jul 16 16:25:59.650978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.649518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:25:59.650978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.649646 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:25:59.650978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.649741 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:25:59.650978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.650190 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b"] Jul 16 16:25:59.654827 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.654806 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vjpd8\"" Jul 16 16:25:59.656416 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.655110 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 16:25:59.656416 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.655120 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 16:25:59.656416 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.655375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npp5l\"" Jul 16 16:25:59.662642 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.662560 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-6g8qg"] Jul 16 16:25:59.666042 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.664426 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-67bqh"] Jul 16 16:25:59.669230 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.669210 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fb97c79fd-4cvvx"] Jul 16 16:25:59.669329 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.669245 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-66dgj"] Jul 16 16:25:59.669385 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.669341 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.673564 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.673539 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k"] Jul 16 16:25:59.673814 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.673690 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.674969 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.674903 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2tpfl\"" Jul 16 16:25:59.675224 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.675201 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 16:25:59.675335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.675162 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 16:25:59.675924 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.675902 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 16:25:59.676006 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.675981 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qw68r\"" Jul 16 16:25:59.676110 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.675913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 16:25:59.676364 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.676343 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z"] Jul 16 16:25:59.681328 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.681307 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 16:25:59.684703 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.684687 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-66dgj"] Jul 16 16:25:59.686488 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.686470 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-67bqh"] Jul 16 16:25:59.697174 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.697301 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28npk\" (UniqueName: \"kubernetes.io/projected/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-kube-api-access-28npk\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.697301 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-ca-trust-extracted\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.697301 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697226 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpv5k\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-kube-api-access-mpv5k\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.697301 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697264 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:25:59.697506 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.697750 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697727 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6675174b-9998-44d2-a629-89c04b7951a1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.697821 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6675174b-9998-44d2-a629-89c04b7951a1-serving-cert\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.697821 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-image-registry-private-configuration\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.697821 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-certificates\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6675174b-9998-44d2-a629-89c04b7951a1-snapshots\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6675174b-9998-44d2-a629-89c04b7951a1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2d1f954f-0913-4444-bf59-dcfd4158b72d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k5cz\" (UniqueName: \"kubernetes.io/projected/2d1f954f-0913-4444-bf59-dcfd4158b72d-kube-api-access-2k5cz\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-trusted-ca\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.697973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697969 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d76s\" (UniqueName: \"kubernetes.io/projected/bde6c88c-8138-4506-8702-1f2ae7a478e5-kube-api-access-9d76s\") pod \"volume-data-source-validator-78495597f6-kmv9b\" (UID: \"bde6c88c-8138-4506-8702-1f2ae7a478e5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" Jul 16 16:25:59.698314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.697988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmnj\" (UniqueName: \"kubernetes.io/projected/6675174b-9998-44d2-a629-89c04b7951a1-kube-api-access-5pmnj\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.698314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.698012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-bound-sa-token\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.698314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.698034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6675174b-9998-44d2-a629-89c04b7951a1-tmp\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.698314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.698074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-installation-pull-secrets\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.698314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.698109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.698314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.698178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4cc34f48-be5e-40c9-8afe-016ef627fda0-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:25:59.798744 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6675174b-9998-44d2-a629-89c04b7951a1-serving-cert\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17e7d176-96bd-4e7b-a4bf-177fe59d7860-tmp-dir\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-image-registry-private-configuration\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdfc70f4-b185-4c08-871b-fb69f667ea17-trusted-ca\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7g7c\" (UniqueName: \"kubernetes.io/projected/e03752d9-97fa-402a-aca0-a5cec5a740f6-kube-api-access-r7g7c\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.798917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6675174b-9998-44d2-a629-89c04b7951a1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798940 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxhn\" (UniqueName: \"kubernetes.io/projected/023daf73-3d61-48e0-b995-695e7696620f-kube-api-access-8nxhn\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-bound-sa-token\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.798984 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2d1f954f-0913-4444-bf59-dcfd4158b72d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799001 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmnj\" (UniqueName: \"kubernetes.io/projected/6675174b-9998-44d2-a629-89c04b7951a1-kube-api-access-5pmnj\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdfc70f4-b185-4c08-871b-fb69f667ea17-config\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4cc34f48-be5e-40c9-8afe-016ef627fda0-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jvqm\" (UniqueName: \"kubernetes.io/projected/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-kube-api-access-7jvqm\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-ca-trust-extracted\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cdfc70f4-b185-4c08-871b-fb69f667ea17-serving-cert\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799174 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpv5k\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-kube-api-access-mpv5k\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799190 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799204 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e03752d9-97fa-402a-aca0-a5cec5a740f6-serving-cert\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.799302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799218 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17e7d176-96bd-4e7b-a4bf-177fe59d7860-config-volume\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799236 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799253 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs24z\" (UniqueName: \"kubernetes.io/projected/17e7d176-96bd-4e7b-a4bf-177fe59d7860-kube-api-access-zs24z\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6675174b-9998-44d2-a629-89c04b7951a1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-certificates\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6675174b-9998-44d2-a629-89c04b7951a1-snapshots\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799352 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e03752d9-97fa-402a-aca0-a5cec5a740f6-config\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k5cz\" (UniqueName: \"kubernetes.io/projected/2d1f954f-0913-4444-bf59-dcfd4158b72d-kube-api-access-2k5cz\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.799615 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.799628 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4cdb78f7-jz5rc: secret "image-registry-tls" not found Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.799696 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls podName:58d2f5b0-84f5-4acb-82b0-05c2f1be4baf nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.299680099 +0000 UTC m=+36.107792106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls") pod "image-registry-b4cdb78f7-jz5rc" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf") : secret "image-registry-tls" not found Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799734 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2d1f954f-0913-4444-bf59-dcfd4158b72d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.799979 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.799820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6675174b-9998-44d2-a629-89c04b7951a1-snapshots\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.800638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.800409 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4cc34f48-be5e-40c9-8afe-016ef627fda0-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:25:59.800638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.800462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6675174b-9998-44d2-a629-89c04b7951a1-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.800760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.800735 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6675174b-9998-44d2-a629-89c04b7951a1-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.800818 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.800778 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-ca-trust-extracted\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.800931 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.800913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.800996 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.800979 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-trusted-ca\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.801051 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801017 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9snj\" (UniqueName: \"kubernetes.io/projected/61a33d14-dc39-4e09-bf09-6081815565e3-kube-api-access-p9snj\") pod \"network-check-source-869dcfcdcd-cq4df\" (UID: \"61a33d14-dc39-4e09-bf09-6081815565e3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" Jul 16 16:25:59.801220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801083 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d76s\" (UniqueName: \"kubernetes.io/projected/bde6c88c-8138-4506-8702-1f2ae7a478e5-kube-api-access-9d76s\") pod \"volume-data-source-validator-78495597f6-kmv9b\" (UID: \"bde6c88c-8138-4506-8702-1f2ae7a478e5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" Jul 16 16:25:59.801220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0271013-b398-4270-8978-4a2e6a37e4b5-cert\") pod \"ingress-canary-66dgj\" (UID: \"f0271013-b398-4270-8978-4a2e6a37e4b5\") " pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.801220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6675174b-9998-44d2-a629-89c04b7951a1-tmp\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.801220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-installation-pull-secrets\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.801220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801193 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8vbg\" (UniqueName: \"kubernetes.io/projected/cdfc70f4-b185-4c08-871b-fb69f667ea17-kube-api-access-s8vbg\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.801220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-stats-auth\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-default-certificate\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801296 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28npk\" (UniqueName: \"kubernetes.io/projected/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-kube-api-access-28npk\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8x29\" (UniqueName: \"kubernetes.io/projected/f0271013-b398-4270-8978-4a2e6a37e4b5-kube-api-access-s8x29\") pod \"ingress-canary-66dgj\" (UID: \"f0271013-b398-4270-8978-4a2e6a37e4b5\") " pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.801532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801491 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-certificates\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.801857 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.801757 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:25:59.801857 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.801808 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls podName:2d1f954f-0913-4444-bf59-dcfd4158b72d nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.30179323 +0000 UTC m=+36.109905225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-66skd" (UID: "2d1f954f-0913-4444-bf59-dcfd4158b72d") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:25:59.801968 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801900 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-trusted-ca\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.802019 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.802007 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 16:25:59.802102 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.802083 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert podName:4cc34f48-be5e-40c9-8afe-016ef627fda0 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.302047902 +0000 UTC m=+36.110159899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-tjjpr" (UID: "4cc34f48-be5e-40c9-8afe-016ef627fda0") : secret "networking-console-plugin-cert" not found Jul 16 16:25:59.802102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.801757 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6675174b-9998-44d2-a629-89c04b7951a1-tmp\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.803480 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.803446 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.803576 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.803558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6675174b-9998-44d2-a629-89c04b7951a1-serving-cert\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.804011 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.803985 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-image-registry-private-configuration\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.804333 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.804313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-installation-pull-secrets\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.826570 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.826540 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmnj\" (UniqueName: \"kubernetes.io/projected/6675174b-9998-44d2-a629-89c04b7951a1-kube-api-access-5pmnj\") pod \"insights-operator-68d9dd94cf-dlqfr\" (UID: \"6675174b-9998-44d2-a629-89c04b7951a1\") " pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.840376 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.840347 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28npk\" (UniqueName: \"kubernetes.io/projected/bf714cea-2ff9-4200-ad09-6c3f7eba5ae4-kube-api-access-28npk\") pod \"kube-storage-version-migrator-operator-799cb7f958-2ks42\" (UID: \"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.840376 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.840379 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d76s\" (UniqueName: \"kubernetes.io/projected/bde6c88c-8138-4506-8702-1f2ae7a478e5-kube-api-access-9d76s\") pod \"volume-data-source-validator-78495597f6-kmv9b\" (UID: \"bde6c88c-8138-4506-8702-1f2ae7a478e5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" Jul 16 16:25:59.841642 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.841621 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpv5k\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-kube-api-access-mpv5k\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.854117 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.854047 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k5cz\" (UniqueName: \"kubernetes.io/projected/2d1f954f-0913-4444-bf59-dcfd4158b72d-kube-api-access-2k5cz\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:25:59.854730 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.854706 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-bound-sa-token\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:25:59.891240 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.891161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" Jul 16 16:25:59.902531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxhn\" (UniqueName: \"kubernetes.io/projected/023daf73-3d61-48e0-b995-695e7696620f-kube-api-access-8nxhn\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:25:59.902660 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdfc70f4-b185-4c08-871b-fb69f667ea17-config\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.902660 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:25:59.902660 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jvqm\" (UniqueName: \"kubernetes.io/projected/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-kube-api-access-7jvqm\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.902660 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902646 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cdfc70f4-b185-4c08-871b-fb69f667ea17-serving-cert\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.902868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.902868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e03752d9-97fa-402a-aca0-a5cec5a740f6-serving-cert\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.902868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17e7d176-96bd-4e7b-a4bf-177fe59d7860-config-volume\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.902868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.902758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs24z\" (UniqueName: \"kubernetes.io/projected/17e7d176-96bd-4e7b-a4bf-177fe59d7860-kube-api-access-zs24z\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.902868 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.902773 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:25:59.902868 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.902854 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls podName:023daf73-3d61-48e0-b995-695e7696620f nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.402833366 +0000 UTC m=+36.210945359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls") pod "cluster-samples-operator-65db858894-rlf8z" (UID: "023daf73-3d61-48e0-b995-695e7696620f") : secret "samples-operator-tls" not found Jul 16 16:25:59.903097 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e03752d9-97fa-402a-aca0-a5cec5a740f6-config\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.903156 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9snj\" (UniqueName: \"kubernetes.io/projected/61a33d14-dc39-4e09-bf09-6081815565e3-kube-api-access-p9snj\") pod \"network-check-source-869dcfcdcd-cq4df\" (UID: \"61a33d14-dc39-4e09-bf09-6081815565e3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" Jul 16 16:25:59.903191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0271013-b398-4270-8978-4a2e6a37e4b5-cert\") pod \"ingress-canary-66dgj\" (UID: \"f0271013-b398-4270-8978-4a2e6a37e4b5\") " pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.903191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8vbg\" (UniqueName: \"kubernetes.io/projected/cdfc70f4-b185-4c08-871b-fb69f667ea17-kube-api-access-s8vbg\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.903272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903205 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-stats-auth\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.903272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903230 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-default-certificate\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.903272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.903272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8x29\" (UniqueName: \"kubernetes.io/projected/f0271013-b398-4270-8978-4a2e6a37e4b5-kube-api-access-s8x29\") pod \"ingress-canary-66dgj\" (UID: \"f0271013-b398-4270-8978-4a2e6a37e4b5\") " pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.903435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17e7d176-96bd-4e7b-a4bf-177fe59d7860-tmp-dir\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.903435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdfc70f4-b185-4c08-871b-fb69f667ea17-trusted-ca\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.903435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903306 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdfc70f4-b185-4c08-871b-fb69f667ea17-config\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.903435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903326 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.903435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7g7c\" (UniqueName: \"kubernetes.io/projected/e03752d9-97fa-402a-aca0-a5cec5a740f6-kube-api-access-r7g7c\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.903435 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.903400 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.403389059 +0000 UTC m=+36.211501051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : configmap references non-existent config key: service-ca.crt Jul 16 16:25:59.903723 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.903703 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17e7d176-96bd-4e7b-a4bf-177fe59d7860-config-volume\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.904487 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.904024 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e03752d9-97fa-402a-aca0-a5cec5a740f6-config\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.904487 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.904135 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:25:59.904487 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.904179 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls podName:17e7d176-96bd-4e7b-a4bf-177fe59d7860 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.404165067 +0000 UTC m=+36.212277076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls") pod "dns-default-67bqh" (UID: "17e7d176-96bd-4e7b-a4bf-177fe59d7860") : secret "dns-default-metrics-tls" not found Jul 16 16:25:59.904487 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.904307 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:25:59.904487 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:25:59.904350 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:00.404336355 +0000 UTC m=+36.212448351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : secret "router-metrics-certs-default" not found Jul 16 16:25:59.905160 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.904858 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17e7d176-96bd-4e7b-a4bf-177fe59d7860-tmp-dir\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.905819 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.905776 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cdfc70f4-b185-4c08-871b-fb69f667ea17-trusted-ca\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.907247 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.907136 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0271013-b398-4270-8978-4a2e6a37e4b5-cert\") pod \"ingress-canary-66dgj\" (UID: \"f0271013-b398-4270-8978-4a2e6a37e4b5\") " pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.907247 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.907162 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-default-certificate\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.908169 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.908151 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-stats-auth\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.908305 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.908276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e03752d9-97fa-402a-aca0-a5cec5a740f6-serving-cert\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.908622 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.908596 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cdfc70f4-b185-4c08-871b-fb69f667ea17-serving-cert\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.919440 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.919413 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8vbg\" (UniqueName: \"kubernetes.io/projected/cdfc70f4-b185-4c08-871b-fb69f667ea17-kube-api-access-s8vbg\") pod \"console-operator-7866497494-6g8qg\" (UID: \"cdfc70f4-b185-4c08-871b-fb69f667ea17\") " pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:25:59.920955 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.920840 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9snj\" (UniqueName: \"kubernetes.io/projected/61a33d14-dc39-4e09-bf09-6081815565e3-kube-api-access-p9snj\") pod \"network-check-source-869dcfcdcd-cq4df\" (UID: \"61a33d14-dc39-4e09-bf09-6081815565e3\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" Jul 16 16:25:59.920955 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.920947 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8x29\" (UniqueName: \"kubernetes.io/projected/f0271013-b398-4270-8978-4a2e6a37e4b5-kube-api-access-s8x29\") pod \"ingress-canary-66dgj\" (UID: \"f0271013-b398-4270-8978-4a2e6a37e4b5\") " pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:25:59.921606 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.921574 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jvqm\" (UniqueName: \"kubernetes.io/projected/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-kube-api-access-7jvqm\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:25:59.922760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.922734 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxhn\" (UniqueName: \"kubernetes.io/projected/023daf73-3d61-48e0-b995-695e7696620f-kube-api-access-8nxhn\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:25:59.923036 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.923010 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs24z\" (UniqueName: \"kubernetes.io/projected/17e7d176-96bd-4e7b-a4bf-177fe59d7860-kube-api-access-zs24z\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:25:59.923409 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.923392 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7g7c\" (UniqueName: \"kubernetes.io/projected/e03752d9-97fa-402a-aca0-a5cec5a740f6-kube-api-access-r7g7c\") pod \"service-ca-operator-7695fc6c8c-r2z8k\" (UID: \"e03752d9-97fa-402a-aca0-a5cec5a740f6\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:25:59.927721 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.927701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" Jul 16 16:25:59.937167 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.937142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" Jul 16 16:25:59.954816 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.954601 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" Jul 16 16:25:59.978416 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:25:59.977039 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:26:00.005291 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.004235 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" Jul 16 16:26:00.051340 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.050861 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-66dgj" Jul 16 16:26:00.122102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.121975 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dlqfr"] Jul 16 16:26:00.151542 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.151442 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42"] Jul 16 16:26:00.159101 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:00.159040 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf714cea_2ff9_4200_ad09_6c3f7eba5ae4.slice/crio-3f0e1c28c8cd68aba2cef33de422c1dcc1aaf453684e53fdf3cdf70764bfeeed WatchSource:0}: Error finding container 3f0e1c28c8cd68aba2cef33de422c1dcc1aaf453684e53fdf3cdf70764bfeeed: Status 404 returned error can't find the container with id 3f0e1c28c8cd68aba2cef33de422c1dcc1aaf453684e53fdf3cdf70764bfeeed Jul 16 16:26:00.163606 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:00.163572 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde6c88c_8138_4506_8702_1f2ae7a478e5.slice/crio-f0e879795dad1bebab819bf4f421d56935ee994f4d07349d53d819b8da5f90b8 WatchSource:0}: Error finding container f0e879795dad1bebab819bf4f421d56935ee994f4d07349d53d819b8da5f90b8: Status 404 returned error can't find the container with id f0e879795dad1bebab819bf4f421d56935ee994f4d07349d53d819b8da5f90b8 Jul 16 16:26:00.169737 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.165038 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b"] Jul 16 16:26:00.189790 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.189751 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df"] Jul 16 16:26:00.194442 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:00.194410 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a33d14_dc39_4e09_bf09_6081815565e3.slice/crio-20fc37247f0829c3e94bfdcbe56269d72edcd7ed474c6d83582dc9b986ffcd38 WatchSource:0}: Error finding container 20fc37247f0829c3e94bfdcbe56269d72edcd7ed474c6d83582dc9b986ffcd38: Status 404 returned error can't find the container with id 20fc37247f0829c3e94bfdcbe56269d72edcd7ed474c6d83582dc9b986ffcd38 Jul 16 16:26:00.220620 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.220580 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k"] Jul 16 16:26:00.223555 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:00.223523 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode03752d9_97fa_402a_aca0_a5cec5a740f6.slice/crio-8aa3c3d8962f19967b3d3c557622c68ea648da1c62993a98cbb31623d2dd76f0 WatchSource:0}: Error finding container 8aa3c3d8962f19967b3d3c557622c68ea648da1c62993a98cbb31623d2dd76f0: Status 404 returned error can't find the container with id 8aa3c3d8962f19967b3d3c557622c68ea648da1c62993a98cbb31623d2dd76f0 Jul 16 16:26:00.245956 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.245929 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-6g8qg"] Jul 16 16:26:00.248898 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:00.248867 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfc70f4_b185_4c08_871b_fb69f667ea17.slice/crio-ff926fc6a2184e9d83335b79330e2a44e0f806f07cacf48ffad2758ad920d82c WatchSource:0}: Error finding container ff926fc6a2184e9d83335b79330e2a44e0f806f07cacf48ffad2758ad920d82c: Status 404 returned error can't find the container with id ff926fc6a2184e9d83335b79330e2a44e0f806f07cacf48ffad2758ad920d82c Jul 16 16:26:00.263370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.263346 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-66dgj"] Jul 16 16:26:00.266431 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:00.266406 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0271013_b398_4270_8978_4a2e6a37e4b5.slice/crio-17ca1263a898df09ea2e5bb11c11d04eb87df7397df3f3fa7dbbf52010a8422d WatchSource:0}: Error finding container 17ca1263a898df09ea2e5bb11c11d04eb87df7397df3f3fa7dbbf52010a8422d: Status 404 returned error can't find the container with id 17ca1263a898df09ea2e5bb11c11d04eb87df7397df3f3fa7dbbf52010a8422d Jul 16 16:26:00.315245 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.315174 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:00.315402 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.315274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:00.315402 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315337 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:00.315402 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315397 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:26:00.315568 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315405 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls podName:2d1f954f-0913-4444-bf59-dcfd4158b72d nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.31538605 +0000 UTC m=+37.123498059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-66skd" (UID: "2d1f954f-0913-4444-bf59-dcfd4158b72d") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:00.315568 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315408 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4cdb78f7-jz5rc: secret "image-registry-tls" not found Jul 16 16:26:00.315568 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315418 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 16:26:00.315568 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315439 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls podName:58d2f5b0-84f5-4acb-82b0-05c2f1be4baf nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.31543024 +0000 UTC m=+37.123542232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls") pod "image-registry-b4cdb78f7-jz5rc" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf") : secret "image-registry-tls" not found Jul 16 16:26:00.315568 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.315473 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert podName:4cc34f48-be5e-40c9-8afe-016ef627fda0 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.315460981 +0000 UTC m=+37.123572974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-tjjpr" (UID: "4cc34f48-be5e-40c9-8afe-016ef627fda0") : secret "networking-console-plugin-cert" not found Jul 16 16:26:00.315568 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.315335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:00.416996 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.416910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:00.416996 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.416976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.417091 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417095 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417118 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.417153 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417166 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls podName:023daf73-3d61-48e0-b995-695e7696620f nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.417149914 +0000 UTC m=+37.225261925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls") pod "cluster-samples-operator-65db858894-rlf8z" (UID: "023daf73-3d61-48e0-b995-695e7696620f") : secret "samples-operator-tls" not found Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417189 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:26:00.417236 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417231 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.417213839 +0000 UTC m=+37.225325834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : secret "router-metrics-certs-default" not found Jul 16 16:26:00.417531 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417260 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.417250623 +0000 UTC m=+37.225362615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : configmap references non-existent config key: service-ca.crt Jul 16 16:26:00.417531 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:00.417288 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls podName:17e7d176-96bd-4e7b-a4bf-177fe59d7860 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:01.417277565 +0000 UTC m=+37.225389557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls") pod "dns-default-67bqh" (UID: "17e7d176-96bd-4e7b-a4bf-177fe59d7860") : secret "dns-default-metrics-tls" not found Jul 16 16:26:00.838453 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.838386 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" event={"ID":"6675174b-9998-44d2-a629-89c04b7951a1","Type":"ContainerStarted","Data":"eda111afe1bdd76e3654025ae472d3eb28bbdf6c4f874fdc5cfd9944c4da5de7"} Jul 16 16:26:00.839807 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.839782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6g8qg" event={"ID":"cdfc70f4-b185-4c08-871b-fb69f667ea17","Type":"ContainerStarted","Data":"ff926fc6a2184e9d83335b79330e2a44e0f806f07cacf48ffad2758ad920d82c"} Jul 16 16:26:00.841267 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.841040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" event={"ID":"61a33d14-dc39-4e09-bf09-6081815565e3","Type":"ContainerStarted","Data":"20fc37247f0829c3e94bfdcbe56269d72edcd7ed474c6d83582dc9b986ffcd38"} Jul 16 16:26:00.842531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.842509 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-66dgj" event={"ID":"f0271013-b398-4270-8978-4a2e6a37e4b5","Type":"ContainerStarted","Data":"17ca1263a898df09ea2e5bb11c11d04eb87df7397df3f3fa7dbbf52010a8422d"} Jul 16 16:26:00.843654 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.843633 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" event={"ID":"bde6c88c-8138-4506-8702-1f2ae7a478e5","Type":"ContainerStarted","Data":"f0e879795dad1bebab819bf4f421d56935ee994f4d07349d53d819b8da5f90b8"} Jul 16 16:26:00.844839 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.844806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" event={"ID":"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4","Type":"ContainerStarted","Data":"3f0e1c28c8cd68aba2cef33de422c1dcc1aaf453684e53fdf3cdf70764bfeeed"} Jul 16 16:26:00.845987 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:00.845968 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" event={"ID":"e03752d9-97fa-402a-aca0-a5cec5a740f6","Type":"ContainerStarted","Data":"8aa3c3d8962f19967b3d3c557622c68ea648da1c62993a98cbb31623d2dd76f0"} Jul 16 16:26:01.327483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.327438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:01.327682 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.327529 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:01.327682 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327611 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 16:26:01.327682 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.327668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:01.327827 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327685 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert podName:4cc34f48-be5e-40c9-8afe-016ef627fda0 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.327665487 +0000 UTC m=+39.135777479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-tjjpr" (UID: "4cc34f48-be5e-40c9-8afe-016ef627fda0") : secret "networking-console-plugin-cert" not found Jul 16 16:26:01.327827 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327687 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:26:01.327827 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327706 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4cdb78f7-jz5rc: secret "image-registry-tls" not found Jul 16 16:26:01.327827 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327743 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:01.327827 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327756 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls podName:58d2f5b0-84f5-4acb-82b0-05c2f1be4baf nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.327740578 +0000 UTC m=+39.135852575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls") pod "image-registry-b4cdb78f7-jz5rc" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf") : secret "image-registry-tls" not found Jul 16 16:26:01.327827 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.327786 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls podName:2d1f954f-0913-4444-bf59-dcfd4158b72d nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.327772451 +0000 UTC m=+39.135884448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-66skd" (UID: "2d1f954f-0913-4444-bf59-dcfd4158b72d") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:01.428559 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.428520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:01.428749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.428580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:01.428749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.428642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:01.428749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:01.428673 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:01.428749 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428693 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:26:01.428966 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428777 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:26:01.428966 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428788 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:26:01.428966 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428781 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.428737068 +0000 UTC m=+39.236849063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : configmap references non-existent config key: service-ca.crt Jul 16 16:26:01.428966 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428847 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls podName:17e7d176-96bd-4e7b-a4bf-177fe59d7860 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.428824081 +0000 UTC m=+39.236936088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls") pod "dns-default-67bqh" (UID: "17e7d176-96bd-4e7b-a4bf-177fe59d7860") : secret "dns-default-metrics-tls" not found Jul 16 16:26:01.428966 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428865 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.428855427 +0000 UTC m=+39.236967424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : secret "router-metrics-certs-default" not found Jul 16 16:26:01.428966 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:01.428882 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls podName:023daf73-3d61-48e0-b995-695e7696620f nodeName:}" failed. No retries permitted until 2026-07-16 16:26:03.428872411 +0000 UTC m=+39.236984417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls") pod "cluster-samples-operator-65db858894-rlf8z" (UID: "023daf73-3d61-48e0-b995-695e7696620f") : secret "samples-operator-tls" not found Jul 16 16:26:02.863379 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:02.862320 2560 generic.go:356] "Generic (PLEG): container finished" podID="b864d35e-dc48-4c12-a205-18fcd6fc6be9" containerID="f7885058dc13055af844f34772a6a8698ae949d515effb95be078aa87d1e032c" exitCode=0 Jul 16 16:26:02.863379 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:02.862427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerDied","Data":"f7885058dc13055af844f34772a6a8698ae949d515effb95be078aa87d1e032c"} Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.348785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.348950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.348996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349154 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349213 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert podName:4cc34f48-be5e-40c9-8afe-016ef627fda0 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.3491959 +0000 UTC m=+43.157307894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-tjjpr" (UID: "4cc34f48-be5e-40c9-8afe-016ef627fda0") : secret "networking-console-plugin-cert" not found Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349611 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349628 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4cdb78f7-jz5rc: secret "image-registry-tls" not found Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349673 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls podName:58d2f5b0-84f5-4acb-82b0-05c2f1be4baf nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.349657183 +0000 UTC m=+43.157769180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls") pod "image-registry-b4cdb78f7-jz5rc" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf") : secret "image-registry-tls" not found Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349732 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:03.349808 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.349769 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls podName:2d1f954f-0913-4444-bf59-dcfd4158b72d nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.34975701 +0000 UTC m=+43.157869001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-66skd" (UID: "2d1f954f-0913-4444-bf59-dcfd4158b72d") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.449488 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.449570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.449631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.449668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.449796 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.449856 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.449837737 +0000 UTC m=+43.257949729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : secret "router-metrics-certs-default" not found Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.449939 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.449926481 +0000 UTC m=+43.258038474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : configmap references non-existent config key: service-ca.crt Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.450007 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.450039 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls podName:17e7d176-96bd-4e7b-a4bf-177fe59d7860 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.450025586 +0000 UTC m=+43.258137579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls") pod "dns-default-67bqh" (UID: "17e7d176-96bd-4e7b-a4bf-177fe59d7860") : secret "dns-default-metrics-tls" not found Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.450111 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:26:03.450205 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:03.450142 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls podName:023daf73-3d61-48e0-b995-695e7696620f nodeName:}" failed. No retries permitted until 2026-07-16 16:26:07.450132061 +0000 UTC m=+43.258244057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls") pod "cluster-samples-operator-65db858894-rlf8z" (UID: "023daf73-3d61-48e0-b995-695e7696620f") : secret "samples-operator-tls" not found Jul 16 16:26:03.873767 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.873727 2560 generic.go:356] "Generic (PLEG): container finished" podID="b864d35e-dc48-4c12-a205-18fcd6fc6be9" containerID="7072e55d3e37448ad12951261db8d6edf7d2db5b6c8ad862d79754657cddd8e5" exitCode=0 Jul 16 16:26:03.874200 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:03.873779 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerDied","Data":"7072e55d3e37448ad12951261db8d6edf7d2db5b6c8ad862d79754657cddd8e5"} Jul 16 16:26:04.883410 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:04.883195 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" event={"ID":"b864d35e-dc48-4c12-a205-18fcd6fc6be9","Type":"ContainerStarted","Data":"cb68ebaaa7ed2bf7ddc4d2fbe2f3f98e16c495b94fbb57e96304f44c44bfd7fb"} Jul 16 16:26:04.884734 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:04.884704 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" event={"ID":"bde6c88c-8138-4506-8702-1f2ae7a478e5","Type":"ContainerStarted","Data":"185701da7a5a52b356a3b1aa0f1d62b4f09bc52b53da8c4bd4e05812105051db"} Jul 16 16:26:04.908901 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:04.908846 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cvmkb" podStartSLOduration=9.114361221 podStartE2EDuration="40.908826152s" totalImagesPullingTime="31.794464931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:04.907445103 +0000 UTC m=+40.715557119" watchObservedRunningTime="2026-07-16 16:26:04.908826152 +0000 UTC m=+40.716938166" Jul 16 16:26:04.926954 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:04.926906 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-kmv9b" podStartSLOduration=23.131782003 podStartE2EDuration="26.926888217s" totalImagesPullingTime="3.795106214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:04.926674642 +0000 UTC m=+40.734786656" watchObservedRunningTime="2026-07-16 16:26:04.926888217 +0000 UTC m=+40.735000233" Jul 16 16:26:07.386272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.386235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.386348 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.386376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386403 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386426 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4cdb78f7-jz5rc: secret "image-registry-tls" not found Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386459 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386480 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386492 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls podName:58d2f5b0-84f5-4acb-82b0-05c2f1be4baf nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.386473712 +0000 UTC m=+51.194585723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls") pod "image-registry-b4cdb78f7-jz5rc" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf") : secret "image-registry-tls" not found Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386507 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert podName:4cc34f48-be5e-40c9-8afe-016ef627fda0 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.386497122 +0000 UTC m=+51.194609114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-tjjpr" (UID: "4cc34f48-be5e-40c9-8afe-016ef627fda0") : secret "networking-console-plugin-cert" not found Jul 16 16:26:07.386782 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.386525 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls podName:2d1f954f-0913-4444-bf59-dcfd4158b72d nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.386512813 +0000 UTC m=+51.194624804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-66skd" (UID: "2d1f954f-0913-4444-bf59-dcfd4158b72d") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:07.487592 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.487552 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.487599 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.487656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487693 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487750 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487751 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:07.487762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487765 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls podName:023daf73-3d61-48e0-b995-695e7696620f nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.487749249 +0000 UTC m=+51.295861240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls") pod "cluster-samples-operator-65db858894-rlf8z" (UID: "023daf73-3d61-48e0-b995-695e7696620f") : secret "samples-operator-tls" not found Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487835 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls podName:17e7d176-96bd-4e7b-a4bf-177fe59d7860 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.487822374 +0000 UTC m=+51.295934369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls") pod "dns-default-67bqh" (UID: "17e7d176-96bd-4e7b-a4bf-177fe59d7860") : secret "dns-default-metrics-tls" not found Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487846 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.487840697 +0000 UTC m=+51.295952689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : secret "router-metrics-certs-default" not found Jul 16 16:26:07.487882 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:07.487857 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.487851817 +0000 UTC m=+51.295963809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : configmap references non-existent config key: service-ca.crt Jul 16 16:26:09.903368 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:09.903337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" event={"ID":"e03752d9-97fa-402a-aca0-a5cec5a740f6","Type":"ContainerStarted","Data":"371756a26573bbcfd32fa948a9a13e33b3fd2f2d08b3270c8d63e05b9b6b5143"} Jul 16 16:26:09.905911 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:09.905881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6g8qg" event={"ID":"cdfc70f4-b185-4c08-871b-fb69f667ea17","Type":"ContainerStarted","Data":"dfc9e4e53d1a6d7daa8bb58763602048a825c6b4f0f33e5d44ab8c4b110b275d"} Jul 16 16:26:09.912219 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:09.911672 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" event={"ID":"61a33d14-dc39-4e09-bf09-6081815565e3","Type":"ContainerStarted","Data":"5162aa7de5fb5a32733d61842cba9f9b0a5993d24bfe06d1563efb80e6dfd6c5"} Jul 16 16:26:09.939908 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:09.939761 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" podStartSLOduration=21.471564042 podStartE2EDuration="30.93974469s" totalImagesPullingTime="9.468180648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:09.938649683 +0000 UTC m=+45.746761711" watchObservedRunningTime="2026-07-16 16:26:09.93974469 +0000 UTC m=+45.747856707" Jul 16 16:26:09.970532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:09.970269 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-cq4df" podStartSLOduration=18.472320171 podStartE2EDuration="27.970250503s" totalImagesPullingTime="9.497930332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:09.967204737 +0000 UTC m=+45.775316753" watchObservedRunningTime="2026-07-16 16:26:09.970250503 +0000 UTC m=+45.778362517" Jul 16 16:26:10.917110 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.917074 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" event={"ID":"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4","Type":"ContainerStarted","Data":"7459763f175b9b37ed9bfb57a257a0db0c548e509e0395a701efa3398a7daeae"} Jul 16 16:26:10.918670 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.918640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" event={"ID":"6675174b-9998-44d2-a629-89c04b7951a1","Type":"ContainerStarted","Data":"fd438916d0f09429866f9c9b96707a45da81beb46fab23c736d5d327c0455a31"} Jul 16 16:26:10.920469 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.920445 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/0.log" Jul 16 16:26:10.920576 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.920487 2560 generic.go:356] "Generic (PLEG): container finished" podID="cdfc70f4-b185-4c08-871b-fb69f667ea17" containerID="dfc9e4e53d1a6d7daa8bb58763602048a825c6b4f0f33e5d44ab8c4b110b275d" exitCode=255 Jul 16 16:26:10.920576 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.920562 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6g8qg" event={"ID":"cdfc70f4-b185-4c08-871b-fb69f667ea17","Type":"ContainerDied","Data":"dfc9e4e53d1a6d7daa8bb58763602048a825c6b4f0f33e5d44ab8c4b110b275d"} Jul 16 16:26:10.920723 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.920707 2560 scope.go:117] "RemoveContainer" containerID="dfc9e4e53d1a6d7daa8bb58763602048a825c6b4f0f33e5d44ab8c4b110b275d" Jul 16 16:26:10.922112 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.922051 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-66dgj" event={"ID":"f0271013-b398-4270-8978-4a2e6a37e4b5","Type":"ContainerStarted","Data":"4777032ea91b5977907e9409560926f3c836ce1eff62b44208bcb7c5cdb5acd6"} Jul 16 16:26:10.936838 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.936794 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" podStartSLOduration=23.405944871 podStartE2EDuration="32.936776516s" totalImagesPullingTime="9.530831645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:10.935690196 +0000 UTC m=+46.743802209" watchObservedRunningTime="2026-07-16 16:26:10.936776516 +0000 UTC m=+46.744888531" Jul 16 16:26:10.987805 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:10.987557 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" podStartSLOduration=23.423783607 podStartE2EDuration="32.987536066s" totalImagesPullingTime="9.563752459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:10.984148314 +0000 UTC m=+46.792260328" watchObservedRunningTime="2026-07-16 16:26:10.987536066 +0000 UTC m=+46.795648082" Jul 16 16:26:11.021430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.021328 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-66dgj" podStartSLOduration=2.595143873 podStartE2EDuration="12.021306899s" totalImagesPullingTime="9.426163026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:11.018543208 +0000 UTC m=+46.826655244" watchObservedRunningTime="2026-07-16 16:26:11.021306899 +0000 UTC m=+46.829418914" Jul 16 16:26:11.452839 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.452808 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2b84v"] Jul 16 16:26:11.475917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.475886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.479925 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.479889 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jn64s\"" Jul 16 16:26:11.480167 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.480154 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 16:26:11.480277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.480212 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 16:26:11.482998 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.482979 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2b84v"] Jul 16 16:26:11.526695 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.526661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.526868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.526708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5rvn\" (UniqueName: \"kubernetes.io/projected/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-api-access-g5rvn\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.526868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.526734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-data-volume\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.526868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.526799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.527000 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.526925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-crio-socket\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.627307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627272 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.627307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5rvn\" (UniqueName: \"kubernetes.io/projected/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-api-access-g5rvn\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.627563 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627344 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-data-volume\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.627563 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627372 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.627563 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:11.627408 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:26:11.627563 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:11.627474 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:12.127455527 +0000 UTC m=+47.935567542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : configmap "kube-rbac-proxy" not found Jul 16 16:26:11.627563 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627525 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-crio-socket\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.627563 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:11.627530 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:26:11.628013 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:11.627600 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:12.127590725 +0000 UTC m=+47.935702716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : secret "insights-runtime-extractor-tls" not found Jul 16 16:26:11.628013 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627700 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-data-volume\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.628013 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.627710 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-crio-socket\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.640362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.640333 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5rvn\" (UniqueName: \"kubernetes.io/projected/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-api-access-g5rvn\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:11.927318 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.927289 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:26:11.927723 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.927702 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/0.log" Jul 16 16:26:11.927779 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.927734 2560 generic.go:356] "Generic (PLEG): container finished" podID="cdfc70f4-b185-4c08-871b-fb69f667ea17" containerID="f0d2392cdc6d43e88305f6d63244c7766955669b0def419d3493c8a476d1f516" exitCode=255 Jul 16 16:26:11.927779 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.927764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6g8qg" event={"ID":"cdfc70f4-b185-4c08-871b-fb69f667ea17","Type":"ContainerDied","Data":"f0d2392cdc6d43e88305f6d63244c7766955669b0def419d3493c8a476d1f516"} Jul 16 16:26:11.927864 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.927826 2560 scope.go:117] "RemoveContainer" containerID="dfc9e4e53d1a6d7daa8bb58763602048a825c6b4f0f33e5d44ab8c4b110b275d" Jul 16 16:26:11.928107 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:11.928089 2560 scope.go:117] "RemoveContainer" containerID="f0d2392cdc6d43e88305f6d63244c7766955669b0def419d3493c8a476d1f516" Jul 16 16:26:11.928291 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:11.928269 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-6g8qg_openshift-console-operator(cdfc70f4-b185-4c08-871b-fb69f667ea17)\"" pod="openshift-console-operator/console-operator-7866497494-6g8qg" podUID="cdfc70f4-b185-4c08-871b-fb69f667ea17" Jul 16 16:26:12.131635 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:12.131586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:12.131635 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:12.131656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:12.131938 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:12.131727 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:26:12.131938 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:12.131741 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:26:12.131938 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:12.131807 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:13.131787861 +0000 UTC m=+48.939899854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : configmap "kube-rbac-proxy" not found Jul 16 16:26:12.131938 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:12.131827 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:13.131819896 +0000 UTC m=+48.939931889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : secret "insights-runtime-extractor-tls" not found Jul 16 16:26:12.932700 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:12.932669 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:26:12.933150 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:12.933052 2560 scope.go:117] "RemoveContainer" containerID="f0d2392cdc6d43e88305f6d63244c7766955669b0def419d3493c8a476d1f516" Jul 16 16:26:12.933302 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:12.933279 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-6g8qg_openshift-console-operator(cdfc70f4-b185-4c08-871b-fb69f667ea17)\"" pod="openshift-console-operator/console-operator-7866497494-6g8qg" podUID="cdfc70f4-b185-4c08-871b-fb69f667ea17" Jul 16 16:26:13.139294 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:13.139257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:13.139489 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:13.139316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:13.139489 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:13.139395 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 16:26:13.139489 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:13.139405 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:26:13.139489 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:13.139470 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.139450147 +0000 UTC m=+50.947562145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : configmap "kube-rbac-proxy" not found Jul 16 16:26:13.139489 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:13.139491 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:15.139481601 +0000 UTC m=+50.947593600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : secret "insights-runtime-extractor-tls" not found Jul 16 16:26:13.860116 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:13.860085 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8h4f9_579023c8-e486-4414-98ab-42b1dbfc20c6/dns-node-resolver/0.log" Jul 16 16:26:14.661043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:14.661008 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7zm9t_edee432b-41a0-4a35-924e-405d84cdb218/node-ca/0.log" Jul 16 16:26:15.160068 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.160004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:15.160243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.160112 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:15.160287 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.160263 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:26:15.160341 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.160330 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:19.160315032 +0000 UTC m=+54.968427028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : secret "insights-runtime-extractor-tls" not found Jul 16 16:26:15.160573 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.160553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:15.463003 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.462967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:15.463188 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.463095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:15.463188 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.463122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:15.463188 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463139 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 16:26:15.463188 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463159 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4cdb78f7-jz5rc: secret "image-registry-tls" not found Jul 16 16:26:15.463342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463214 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls podName:58d2f5b0-84f5-4acb-82b0-05c2f1be4baf nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.463198624 +0000 UTC m=+67.271310620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls") pod "image-registry-b4cdb78f7-jz5rc" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf") : secret "image-registry-tls" not found Jul 16 16:26:15.463342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463220 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 16:26:15.463342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463222 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:15.463342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463259 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert podName:4cc34f48-be5e-40c9-8afe-016ef627fda0 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.463248353 +0000 UTC m=+67.271360345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-tjjpr" (UID: "4cc34f48-be5e-40c9-8afe-016ef627fda0") : secret "networking-console-plugin-cert" not found Jul 16 16:26:15.463342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.463271 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls podName:2d1f954f-0913-4444-bf59-dcfd4158b72d nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.463265746 +0000 UTC m=+67.271377737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-66skd" (UID: "2d1f954f-0913-4444-bf59-dcfd4158b72d") : secret "cluster-monitoring-operator-tls" not found Jul 16 16:26:15.563508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.563470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:15.563687 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.563536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:15.563687 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.563587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:15.563687 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563612 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 16:26:15.563687 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.563621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:15.563687 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563668 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls podName:17e7d176-96bd-4e7b-a4bf-177fe59d7860 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.563653814 +0000 UTC m=+67.371765805 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls") pod "dns-default-67bqh" (UID: "17e7d176-96bd-4e7b-a4bf-177fe59d7860") : secret "dns-default-metrics-tls" not found Jul 16 16:26:15.563878 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563707 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.56369104 +0000 UTC m=+67.371803056 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : configmap references non-existent config key: service-ca.crt Jul 16 16:26:15.563878 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563705 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 16:26:15.563878 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563711 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 16:26:15.563878 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563764 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls podName:023daf73-3d61-48e0-b995-695e7696620f nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.56375243 +0000 UTC m=+67.371864422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls") pod "cluster-samples-operator-65db858894-rlf8z" (UID: "023daf73-3d61-48e0-b995-695e7696620f") : secret "samples-operator-tls" not found Jul 16 16:26:15.563878 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:15.563801 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs podName:2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:31.563789319 +0000 UTC m=+67.371901314 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs") pod "router-default-7fb97c79fd-4cvvx" (UID: "2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87") : secret "router-metrics-certs-default" not found Jul 16 16:26:15.668600 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:15.668572 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-66dgj_f0271013-b398-4270-8978-4a2e6a37e4b5/serve-healthcheck-canary/0.log" Jul 16 16:26:16.262943 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:16.262906 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2ks42_bf714cea-2ff9-4200-ad09-6c3f7eba5ae4/kube-storage-version-migrator-operator/0.log" Jul 16 16:26:19.193711 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:19.193670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:19.194198 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:19.193825 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 16:26:19.194198 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:19.193890 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls podName:6c2547e7-6a36-4bcb-bfbf-f9422a5c0527 nodeName:}" failed. No retries permitted until 2026-07-16 16:26:27.193874319 +0000 UTC m=+63.001986311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2b84v" (UID: "6c2547e7-6a36-4bcb-bfbf-f9422a5c0527") : secret "insights-runtime-extractor-tls" not found Jul 16 16:26:19.977611 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:19.977570 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:26:19.977917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:19.977630 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:26:19.978116 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:19.978098 2560 scope.go:117] "RemoveContainer" containerID="f0d2392cdc6d43e88305f6d63244c7766955669b0def419d3493c8a476d1f516" Jul 16 16:26:19.978308 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:19.978286 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-6g8qg_openshift-console-operator(cdfc70f4-b185-4c08-871b-fb69f667ea17)\"" pod="openshift-console-operator/console-operator-7866497494-6g8qg" podUID="cdfc70f4-b185-4c08-871b-fb69f667ea17" Jul 16 16:26:25.834130 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:25.834100 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4wlc2" Jul 16 16:26:27.270916 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:27.270878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:27.273315 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:27.273292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2547e7-6a36-4bcb-bfbf-f9422a5c0527-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2b84v\" (UID: \"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527\") " pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:27.389801 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:27.389768 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jn64s\"" Jul 16 16:26:27.397139 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:27.397099 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2b84v" Jul 16 16:26:27.549800 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:27.549723 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2b84v"] Jul 16 16:26:27.552573 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:27.552547 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2547e7_6a36_4bcb_bfbf_f9422a5c0527.slice/crio-ded3ee04250049a46325d1619f2433cd87444f28b687382bdc311b9950f217d8 WatchSource:0}: Error finding container ded3ee04250049a46325d1619f2433cd87444f28b687382bdc311b9950f217d8: Status 404 returned error can't find the container with id ded3ee04250049a46325d1619f2433cd87444f28b687382bdc311b9950f217d8 Jul 16 16:26:27.985336 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:27.985298 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2b84v" event={"ID":"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527","Type":"ContainerStarted","Data":"ded3ee04250049a46325d1619f2433cd87444f28b687382bdc311b9950f217d8"} Jul 16 16:26:30.302333 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.302297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:26:30.304930 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.304686 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 16:26:30.315994 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.315966 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e86a25b1-e8e9-4b5d-b2c6-4011245d5870-metrics-certs\") pod \"network-metrics-daemon-lqvdt\" (UID: \"e86a25b1-e8e9-4b5d-b2c6-4011245d5870\") " pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:26:30.341459 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.341434 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-npp5l\"" Jul 16 16:26:30.348436 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.348412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqvdt" Jul 16 16:26:30.503901 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.503867 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:26:30.504115 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.503908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:26:30.506906 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.506878 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bnvv\" (UniqueName: \"kubernetes.io/projected/4406ffec-a202-45c2-98bb-c83cc037e173-kube-api-access-4bnvv\") pod \"network-check-target-hzbgf\" (UID: \"4406ffec-a202-45c2-98bb-c83cc037e173\") " pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:26:30.507490 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.507469 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 16:26:30.516798 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.516772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ffb1bb7-d510-455e-828d-88887fa3c651-original-pull-secret\") pod \"global-pull-secret-syncer-nsqs2\" (UID: \"1ffb1bb7-d510-455e-828d-88887fa3c651\") " pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:26:30.617902 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.617825 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vjpd8\"" Jul 16 16:26:30.621366 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.621345 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:26:30.645103 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:30.645050 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsqs2" Jul 16 16:26:31.512651 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.512616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:31.513101 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.512665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:31.513101 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.512860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:31.515437 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.515409 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4cc34f48-be5e-40c9-8afe-016ef627fda0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tjjpr\" (UID: \"4cc34f48-be5e-40c9-8afe-016ef627fda0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:31.516327 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.515792 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d1f954f-0913-4444-bf59-dcfd4158b72d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-66skd\" (UID: \"2d1f954f-0913-4444-bf59-dcfd4158b72d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:31.516327 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.515820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"image-registry-b4cdb78f7-jz5rc\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:31.613948 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.613904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:31.614163 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.613964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:31.614163 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.614030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:31.614163 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.614091 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:31.614811 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.614761 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-service-ca-bundle\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:31.616936 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.616908 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87-metrics-certs\") pod \"router-default-7fb97c79fd-4cvvx\" (UID: \"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87\") " pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:31.617046 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.616958 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17e7d176-96bd-4e7b-a4bf-177fe59d7860-metrics-tls\") pod \"dns-default-67bqh\" (UID: \"17e7d176-96bd-4e7b-a4bf-177fe59d7860\") " pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:31.617046 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.617008 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/023daf73-3d61-48e0-b995-695e7696620f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-rlf8z\" (UID: \"023daf73-3d61-48e0-b995-695e7696620f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:31.684939 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.684909 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpvsj\"" Jul 16 16:26:31.693189 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.693169 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:31.700770 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.700740 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-v7cw2\"" Jul 16 16:26:31.708492 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.708467 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" Jul 16 16:26:31.747809 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.747782 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fw2sx\"" Jul 16 16:26:31.753553 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.753526 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" Jul 16 16:26:31.764837 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.764761 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nh98m\"" Jul 16 16:26:31.773499 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.773469 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:31.790551 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.790521 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2tpfl\"" Jul 16 16:26:31.797451 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.797426 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vplzt\"" Jul 16 16:26:31.798444 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.798422 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:31.806292 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:31.806262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" Jul 16 16:26:32.810120 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:32.810085 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nsqs2"] Jul 16 16:26:32.854883 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:32.854843 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd"] Jul 16 16:26:32.856204 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:32.856147 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqvdt"] Jul 16 16:26:32.886622 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:32.886593 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fb97c79fd-4cvvx"] Jul 16 16:26:32.887931 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:32.887908 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-67bqh"] Jul 16 16:26:32.906870 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:32.906837 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffb1bb7_d510_455e_828d_88887fa3c651.slice/crio-8ecf857789d7cdcf457d858a9b9b5dd41b1295f5251a41d88eaffe8d04215f41 WatchSource:0}: Error finding container 8ecf857789d7cdcf457d858a9b9b5dd41b1295f5251a41d88eaffe8d04215f41: Status 404 returned error can't find the container with id 8ecf857789d7cdcf457d858a9b9b5dd41b1295f5251a41d88eaffe8d04215f41 Jul 16 16:26:32.907694 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:32.907657 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1f954f_0913_4444_bf59_dcfd4158b72d.slice/crio-1b82fc3a59def78d9c9325c6f7608b71d2ccba2c5f820fb8ab92b19e99f2503f WatchSource:0}: Error finding container 1b82fc3a59def78d9c9325c6f7608b71d2ccba2c5f820fb8ab92b19e99f2503f: Status 404 returned error can't find the container with id 1b82fc3a59def78d9c9325c6f7608b71d2ccba2c5f820fb8ab92b19e99f2503f Jul 16 16:26:32.908246 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:32.908135 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86a25b1_e8e9_4b5d_b2c6_4011245d5870.slice/crio-eb38bf824ab0b2b8c501fcf58df188fd48118f43e2291a34c6bf38c0ececa267 WatchSource:0}: Error finding container eb38bf824ab0b2b8c501fcf58df188fd48118f43e2291a34c6bf38c0ececa267: Status 404 returned error can't find the container with id eb38bf824ab0b2b8c501fcf58df188fd48118f43e2291a34c6bf38c0ececa267 Jul 16 16:26:32.909596 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:32.909580 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2c29ca_b4a5_43cb_ba1d_e7b8e98a8f87.slice/crio-cd85ec36b56584b3cd0565ae270b5ac4f4fdfc7c1606ab338ddaf5ef604d2da9 WatchSource:0}: Error finding container cd85ec36b56584b3cd0565ae270b5ac4f4fdfc7c1606ab338ddaf5ef604d2da9: Status 404 returned error can't find the container with id cd85ec36b56584b3cd0565ae270b5ac4f4fdfc7c1606ab338ddaf5ef604d2da9 Jul 16 16:26:32.910233 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:32.910211 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17e7d176_96bd_4e7b_a4bf_177fe59d7860.slice/crio-b436072c74e55392c19b0a34195daa6897104a76d1af12ef1a319bf054336b78 WatchSource:0}: Error finding container b436072c74e55392c19b0a34195daa6897104a76d1af12ef1a319bf054336b78: Status 404 returned error can't find the container with id b436072c74e55392c19b0a34195daa6897104a76d1af12ef1a319bf054336b78 Jul 16 16:26:33.005820 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.005785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67bqh" event={"ID":"17e7d176-96bd-4e7b-a4bf-177fe59d7860","Type":"ContainerStarted","Data":"b436072c74e55392c19b0a34195daa6897104a76d1af12ef1a319bf054336b78"} Jul 16 16:26:33.007177 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.007153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqvdt" event={"ID":"e86a25b1-e8e9-4b5d-b2c6-4011245d5870","Type":"ContainerStarted","Data":"eb38bf824ab0b2b8c501fcf58df188fd48118f43e2291a34c6bf38c0ececa267"} Jul 16 16:26:33.008689 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.008486 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" event={"ID":"2d1f954f-0913-4444-bf59-dcfd4158b72d","Type":"ContainerStarted","Data":"1b82fc3a59def78d9c9325c6f7608b71d2ccba2c5f820fb8ab92b19e99f2503f"} Jul 16 16:26:33.009935 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.009915 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nsqs2" event={"ID":"1ffb1bb7-d510-455e-828d-88887fa3c651","Type":"ContainerStarted","Data":"8ecf857789d7cdcf457d858a9b9b5dd41b1295f5251a41d88eaffe8d04215f41"} Jul 16 16:26:33.010944 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.010895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" event={"ID":"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87","Type":"ContainerStarted","Data":"cd85ec36b56584b3cd0565ae270b5ac4f4fdfc7c1606ab338ddaf5ef604d2da9"} Jul 16 16:26:33.100984 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.100875 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzbgf"] Jul 16 16:26:33.102009 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.101958 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z"] Jul 16 16:26:33.103321 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.103280 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b4cdb78f7-jz5rc"] Jul 16 16:26:33.104666 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:33.104539 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4406ffec_a202_45c2_98bb_c83cc037e173.slice/crio-cd1f6d8102a2fbacdaff6279e27890bba77c4d36717cd48e44d5980d219c194f WatchSource:0}: Error finding container cd1f6d8102a2fbacdaff6279e27890bba77c4d36717cd48e44d5980d219c194f: Status 404 returned error can't find the container with id cd1f6d8102a2fbacdaff6279e27890bba77c4d36717cd48e44d5980d219c194f Jul 16 16:26:33.105344 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:33.105082 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d2f5b0_84f5_4acb_82b0_05c2f1be4baf.slice/crio-716cfa62fab0954613802f8587ffb395375f9784f9b5b97545d5f07cc63c2392 WatchSource:0}: Error finding container 716cfa62fab0954613802f8587ffb395375f9784f9b5b97545d5f07cc63c2392: Status 404 returned error can't find the container with id 716cfa62fab0954613802f8587ffb395375f9784f9b5b97545d5f07cc63c2392 Jul 16 16:26:33.111918 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.111889 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tjjpr"] Jul 16 16:26:33.649799 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:33.649769 2560 scope.go:117] "RemoveContainer" containerID="f0d2392cdc6d43e88305f6d63244c7766955669b0def419d3493c8a476d1f516" Jul 16 16:26:34.032022 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.029443 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:26:34.032022 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.029541 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6g8qg" event={"ID":"cdfc70f4-b185-4c08-871b-fb69f667ea17","Type":"ContainerStarted","Data":"6ea0cc4b56115bd80830025cacc53710ad27a22ba3a142075cfafff075fce5ce"} Jul 16 16:26:34.032022 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.030842 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:26:34.042177 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.042142 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2b84v" event={"ID":"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527","Type":"ContainerStarted","Data":"346abd06839cde12fa078452d00177b4457b1b2da782e531bb53f3dc01fc1d2f"} Jul 16 16:26:34.046338 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.046306 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" event={"ID":"023daf73-3d61-48e0-b995-695e7696620f","Type":"ContainerStarted","Data":"fe7d2f18676f60885274bed4f794e8b33b4055ffd22ba4709939bfa8ef369180"} Jul 16 16:26:34.048891 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.048865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" event={"ID":"2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87","Type":"ContainerStarted","Data":"36264fa8fd4507a3d5590e3db0cc46b57be671dc5771e31414529ee08317c4d2"} Jul 16 16:26:34.054625 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.054563 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" event={"ID":"4cc34f48-be5e-40c9-8afe-016ef627fda0","Type":"ContainerStarted","Data":"9a5ac5c2f2de8e35fc32d851eb9217ba0676a1b6751682c55c0454c3deac2002"} Jul 16 16:26:34.061263 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.060653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" event={"ID":"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf","Type":"ContainerStarted","Data":"07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8"} Jul 16 16:26:34.061263 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.060693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" event={"ID":"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf","Type":"ContainerStarted","Data":"716cfa62fab0954613802f8587ffb395375f9784f9b5b97545d5f07cc63c2392"} Jul 16 16:26:34.061263 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.060870 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:34.063915 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.063888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzbgf" event={"ID":"4406ffec-a202-45c2-98bb-c83cc037e173","Type":"ContainerStarted","Data":"e005bc86b6e5094c904bf20c191b7616c2f5caf39d4c5b904f3d336b5b270a56"} Jul 16 16:26:34.064032 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.063937 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzbgf" event={"ID":"4406ffec-a202-45c2-98bb-c83cc037e173","Type":"ContainerStarted","Data":"cd1f6d8102a2fbacdaff6279e27890bba77c4d36717cd48e44d5980d219c194f"} Jul 16 16:26:34.064439 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.064387 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:26:34.092795 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.091499 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-6g8qg" podStartSLOduration=45.647634688 podStartE2EDuration="55.091478879s" totalImagesPullingTime="9.443844191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:34.085180201 +0000 UTC m=+69.893292221" watchObservedRunningTime="2026-07-16 16:26:34.091478879 +0000 UTC m=+69.899590896" Jul 16 16:26:34.148306 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.148249 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" podStartSLOduration=69.148232309 podStartE2EDuration="1m9.148232309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:34.148186642 +0000 UTC m=+69.956298661" watchObservedRunningTime="2026-07-16 16:26:34.148232309 +0000 UTC m=+69.956344304" Jul 16 16:26:34.223602 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.221979 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hzbgf" podStartSLOduration=70.221957775 podStartE2EDuration="1m10.221957775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:34.221248467 +0000 UTC m=+70.029360482" watchObservedRunningTime="2026-07-16 16:26:34.221957775 +0000 UTC m=+70.030069791" Jul 16 16:26:34.223602 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.223394 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" podStartSLOduration=55.223375993 podStartE2EDuration="55.223375993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:34.189075484 +0000 UTC m=+69.997187498" watchObservedRunningTime="2026-07-16 16:26:34.223375993 +0000 UTC m=+70.031488009" Jul 16 16:26:34.774328 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.774281 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:34.777884 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:34.777854 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:35.032387 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:35.032287 2560 patch_prober.go:28] interesting pod/console-operator-7866497494-6g8qg container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 16:26:35.032816 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:35.032374 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7866497494-6g8qg" podUID="cdfc70f4-b185-4c08-871b-fb69f667ea17" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 16 16:26:35.071747 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:35.071683 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:35.073732 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:35.073536 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7fb97c79fd-4cvvx" Jul 16 16:26:35.079767 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:35.079711 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-6g8qg" Jul 16 16:26:37.407337 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.407300 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74"] Jul 16 16:26:37.413395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.413371 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74"] Jul 16 16:26:37.413562 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.413474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.417051 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.417030 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 16 16:26:37.417204 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.417139 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 16:26:37.417204 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.417174 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 16:26:37.417512 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.417495 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 16:26:37.417614 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.417496 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7grcx\"" Jul 16 16:26:37.441006 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.440974 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-gwrlf"] Jul 16 16:26:37.445648 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.445624 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:26:37.454928 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.454907 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 16:26:37.455405 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.455386 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-t4sjp\"" Jul 16 16:26:37.455496 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.455442 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 16:26:37.480004 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.479930 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gwrlf"] Jul 16 16:26:37.576119 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.576015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34de73a7-d5c1-47fe-aa06-8442608d4155-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74\" (UID: \"34de73a7-d5c1-47fe-aa06-8442608d4155\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.576119 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.576081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25qgz\" (UniqueName: \"kubernetes.io/projected/34de73a7-d5c1-47fe-aa06-8442608d4155-kube-api-access-25qgz\") pod \"managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74\" (UID: \"34de73a7-d5c1-47fe-aa06-8442608d4155\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.576362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.576140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc899\" (UniqueName: \"kubernetes.io/projected/774b6eef-91d7-4267-8f74-f538e73313d1-kube-api-access-rc899\") pod \"downloads-68c9cbb497-gwrlf\" (UID: \"774b6eef-91d7-4267-8f74-f538e73313d1\") " pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:26:37.677072 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.677018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25qgz\" (UniqueName: \"kubernetes.io/projected/34de73a7-d5c1-47fe-aa06-8442608d4155-kube-api-access-25qgz\") pod \"managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74\" (UID: \"34de73a7-d5c1-47fe-aa06-8442608d4155\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.677255 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.677114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc899\" (UniqueName: \"kubernetes.io/projected/774b6eef-91d7-4267-8f74-f538e73313d1-kube-api-access-rc899\") pod \"downloads-68c9cbb497-gwrlf\" (UID: \"774b6eef-91d7-4267-8f74-f538e73313d1\") " pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:26:37.677255 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.677185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34de73a7-d5c1-47fe-aa06-8442608d4155-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74\" (UID: \"34de73a7-d5c1-47fe-aa06-8442608d4155\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.680180 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.680155 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34de73a7-d5c1-47fe-aa06-8442608d4155-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74\" (UID: \"34de73a7-d5c1-47fe-aa06-8442608d4155\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.699532 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.699497 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25qgz\" (UniqueName: \"kubernetes.io/projected/34de73a7-d5c1-47fe-aa06-8442608d4155-kube-api-access-25qgz\") pod \"managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74\" (UID: \"34de73a7-d5c1-47fe-aa06-8442608d4155\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.701174 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.701150 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc899\" (UniqueName: \"kubernetes.io/projected/774b6eef-91d7-4267-8f74-f538e73313d1-kube-api-access-rc899\") pod \"downloads-68c9cbb497-gwrlf\" (UID: \"774b6eef-91d7-4267-8f74-f538e73313d1\") " pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:26:37.747631 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.747598 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" Jul 16 16:26:37.756726 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:37.756701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:26:41.241205 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:41.241170 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gwrlf"] Jul 16 16:26:41.326722 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:41.326682 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod774b6eef_91d7_4267_8f74_f538e73313d1.slice/crio-f8fe6068797e3829cc8620a957fb66a7574922ed25ead36fe78651e2b90b8595 WatchSource:0}: Error finding container f8fe6068797e3829cc8620a957fb66a7574922ed25ead36fe78651e2b90b8595: Status 404 returned error can't find the container with id f8fe6068797e3829cc8620a957fb66a7574922ed25ead36fe78651e2b90b8595 Jul 16 16:26:41.479144 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:41.478956 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74"] Jul 16 16:26:41.483865 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:26:41.483824 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34de73a7_d5c1_47fe_aa06_8442608d4155.slice/crio-e61b887d4bf30530b10d3d44d2e2d769297ce4da9b7e40357e0c255bf5672642 WatchSource:0}: Error finding container e61b887d4bf30530b10d3d44d2e2d769297ce4da9b7e40357e0c255bf5672642: Status 404 returned error can't find the container with id e61b887d4bf30530b10d3d44d2e2d769297ce4da9b7e40357e0c255bf5672642 Jul 16 16:26:42.099302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.099211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" event={"ID":"2d1f954f-0913-4444-bf59-dcfd4158b72d","Type":"ContainerStarted","Data":"7d22c0023fd0ea7f8c114290222c5e9d6b319e5d3c0dcfb04541cd2d2a4a4937"} Jul 16 16:26:42.100665 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.100636 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gwrlf" event={"ID":"774b6eef-91d7-4267-8f74-f538e73313d1","Type":"ContainerStarted","Data":"f8fe6068797e3829cc8620a957fb66a7574922ed25ead36fe78651e2b90b8595"} Jul 16 16:26:42.102670 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.102647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqvdt" event={"ID":"e86a25b1-e8e9-4b5d-b2c6-4011245d5870","Type":"ContainerStarted","Data":"80a195c1965d1ecad574cf4f9947eaa6d5851fdb2eff1e146e08d02e8830ec5c"} Jul 16 16:26:42.102766 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.102690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqvdt" event={"ID":"e86a25b1-e8e9-4b5d-b2c6-4011245d5870","Type":"ContainerStarted","Data":"c49b90e833c03c3cd1b4fbb0dc684fb7a8676781db1b2baf5ac213bec7b95fec"} Jul 16 16:26:42.105289 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.104824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" event={"ID":"4cc34f48-be5e-40c9-8afe-016ef627fda0","Type":"ContainerStarted","Data":"7ac0f5df2b7e4535269e05293678f196e87d30e9cc475308b422789ef8a15428"} Jul 16 16:26:42.106451 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.106397 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" event={"ID":"34de73a7-d5c1-47fe-aa06-8442608d4155","Type":"ContainerStarted","Data":"e61b887d4bf30530b10d3d44d2e2d769297ce4da9b7e40357e0c255bf5672642"} Jul 16 16:26:42.108586 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.108501 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67bqh" event={"ID":"17e7d176-96bd-4e7b-a4bf-177fe59d7860","Type":"ContainerStarted","Data":"19621b96e2ba62655de4793761b6587831c36d6b416e6d914d3b661b930bdf71"} Jul 16 16:26:42.108586 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.108543 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67bqh" event={"ID":"17e7d176-96bd-4e7b-a4bf-177fe59d7860","Type":"ContainerStarted","Data":"c15a7e54793ce9e80d16af7929c85002ed30c7fdd3d785f1aeecfba03918caca"} Jul 16 16:26:42.108940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.108781 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:42.110756 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.110736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2b84v" event={"ID":"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527","Type":"ContainerStarted","Data":"cf5b05651cba1b5b3e74d5dd61d83f0f4e1609b7b221650e5cc24a1a6f2dabb1"} Jul 16 16:26:42.112857 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.112832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" event={"ID":"023daf73-3d61-48e0-b995-695e7696620f","Type":"ContainerStarted","Data":"ea21fc2237347a81f4d16dabb9c2bb3a21e66eeb048570274d3b3fdfd4ce35d0"} Jul 16 16:26:42.112953 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.112875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" event={"ID":"023daf73-3d61-48e0-b995-695e7696620f","Type":"ContainerStarted","Data":"28aa316831bc9fbe555643c22f47a6255ebdfa520430bb151e0dfd868517aa3a"} Jul 16 16:26:42.114423 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.114403 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nsqs2" event={"ID":"1ffb1bb7-d510-455e-828d-88887fa3c651","Type":"ContainerStarted","Data":"4add25317d8b8a36ea3f6181618c33f1f0b2fa5dddf929713e8b4db5c680b576"} Jul 16 16:26:42.133676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.133628 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-66skd" podStartSLOduration=56.154786342 podStartE2EDuration="1m4.13361364s" totalImagesPullingTime="7.978827298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:42.132977672 +0000 UTC m=+77.941089687" watchObservedRunningTime="2026-07-16 16:26:42.13361364 +0000 UTC m=+77.941725647" Jul 16 16:26:42.204682 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.204634 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nsqs2" podStartSLOduration=68.774138063 podStartE2EDuration="1m17.204618653s" totalImagesPullingTime="8.43048059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:42.204246011 +0000 UTC m=+78.012358026" watchObservedRunningTime="2026-07-16 16:26:42.204618653 +0000 UTC m=+78.012730667" Jul 16 16:26:42.204950 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.204915 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-tjjpr" podStartSLOduration=48.085447915 podStartE2EDuration="55.204903872s" totalImagesPullingTime="7.119455957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:42.169267394 +0000 UTC m=+77.977379409" watchObservedRunningTime="2026-07-16 16:26:42.204903872 +0000 UTC m=+78.013015888" Jul 16 16:26:42.232775 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.232722 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-rlf8z" podStartSLOduration=55.0546111 podStartE2EDuration="1m3.232706655s" totalImagesPullingTime="8.178095555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:42.230785475 +0000 UTC m=+78.038897506" watchObservedRunningTime="2026-07-16 16:26:42.232706655 +0000 UTC m=+78.040818694" Jul 16 16:26:42.287352 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.287292 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-67bqh" podStartSLOduration=35.308439683 podStartE2EDuration="43.287271079s" totalImagesPullingTime="7.978831396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:42.284993065 +0000 UTC m=+78.093105081" watchObservedRunningTime="2026-07-16 16:26:42.287271079 +0000 UTC m=+78.095383094" Jul 16 16:26:42.348477 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:42.348427 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lqvdt" podStartSLOduration=71.089272772 podStartE2EDuration="1m18.348411982s" totalImagesPullingTime="7.25913921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:42.345150272 +0000 UTC m=+78.153262301" watchObservedRunningTime="2026-07-16 16:26:42.348411982 +0000 UTC m=+78.156523973" Jul 16 16:26:48.146252 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:48.146214 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2b84v" event={"ID":"6c2547e7-6a36-4bcb-bfbf-f9422a5c0527","Type":"ContainerStarted","Data":"b0a7360ab765f714ccedd325a2a298c6818b940365cac235482914ae47571e01"} Jul 16 16:26:48.147639 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:48.147615 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" event={"ID":"34de73a7-d5c1-47fe-aa06-8442608d4155","Type":"ContainerStarted","Data":"3ac1dcfa782357c416225031fb1c069e3d605486be877b2e8e62f30a7cb9a267"} Jul 16 16:26:48.170340 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:48.170276 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2b84v" podStartSLOduration=17.839186021 podStartE2EDuration="37.17025656s" totalImagesPullingTime="19.331070539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:26:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:48.168681523 +0000 UTC m=+83.976793542" watchObservedRunningTime="2026-07-16 16:26:48.17025656 +0000 UTC m=+83.978368610" Jul 16 16:26:48.186975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:48.186902 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5bc8d9ccdb-bgf74" podStartSLOduration=5.500901774 podStartE2EDuration="11.186879084s" totalImagesPullingTime="5.68597731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:26:48.186429381 +0000 UTC m=+83.994541400" watchObservedRunningTime="2026-07-16 16:26:48.186879084 +0000 UTC m=+83.994991102" Jul 16 16:26:51.697931 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:51.697899 2560 patch_prober.go:28] interesting pod/image-registry-b4cdb78f7-jz5rc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 16:26:51.698408 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:51.697952 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" podUID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 16:26:52.123262 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:52.123179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-67bqh" Jul 16 16:26:55.077483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:55.077457 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:26:59.536251 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.536215 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7wsdw"] Jul 16 16:26:59.687518 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.687487 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.690019 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.689990 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 16:26:59.690177 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.690046 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 16:26:59.690236 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.690188 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 16:26:59.690468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.690447 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kngrv\"" Jul 16 16:26:59.690599 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.690514 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 16:26:59.779913 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.779866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqxt8\" (UniqueName: \"kubernetes.io/projected/b40a9b93-01a7-422f-b912-6b9c08a6d64e-kube-api-access-tqxt8\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.779913 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.779916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-textfile\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.779952 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-root\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.779979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.780116 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-tls\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.780145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40a9b93-01a7-422f-b912-6b9c08a6d64e-metrics-client-ca\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.780189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780452 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.780242 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-wtmp\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.780452 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.780288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-sys\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqxt8\" (UniqueName: \"kubernetes.io/projected/b40a9b93-01a7-422f-b912-6b9c08a6d64e-kube-api-access-tqxt8\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-textfile\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-root\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881732 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-tls\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40a9b93-01a7-422f-b912-6b9c08a6d64e-metrics-client-ca\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-wtmp\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-sys\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.881940 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:59.881880 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 16 16:26:59.882335 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:26:59.881967 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-tls podName:b40a9b93-01a7-422f-b912-6b9c08a6d64e nodeName:}" failed. No retries permitted until 2026-07-16 16:27:00.38194265 +0000 UTC m=+96.190054644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-tls") pod "node-exporter-7wsdw" (UID: "b40a9b93-01a7-422f-b912-6b9c08a6d64e") : secret "node-exporter-tls" not found Jul 16 16:26:59.882335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-root\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.882335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881998 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-textfile\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.882335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.881941 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-sys\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.882335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.882147 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-wtmp\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.882534 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.882357 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.882534 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.882349 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40a9b93-01a7-422f-b912-6b9c08a6d64e-metrics-client-ca\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.884875 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.884854 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:26:59.894960 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:26:59.894933 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqxt8\" (UniqueName: \"kubernetes.io/projected/b40a9b93-01a7-422f-b912-6b9c08a6d64e-kube-api-access-tqxt8\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:27:00.387437 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:00.387397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-tls\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:27:00.390195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:00.390147 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40a9b93-01a7-422f-b912-6b9c08a6d64e-node-exporter-tls\") pod \"node-exporter-7wsdw\" (UID: \"b40a9b93-01a7-422f-b912-6b9c08a6d64e\") " pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:27:00.599508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:00.599226 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7wsdw" Jul 16 16:27:02.690463 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:27:02.690430 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40a9b93_01a7_422f_b912_6b9c08a6d64e.slice/crio-b39861851c69a41ddf464f94f49954301634d0fdc724471cbc38fef0c37a398d WatchSource:0}: Error finding container b39861851c69a41ddf464f94f49954301634d0fdc724471cbc38fef0c37a398d: Status 404 returned error can't find the container with id b39861851c69a41ddf464f94f49954301634d0fdc724471cbc38fef0c37a398d Jul 16 16:27:03.215282 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.214704 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gwrlf" event={"ID":"774b6eef-91d7-4267-8f74-f538e73313d1","Type":"ContainerStarted","Data":"ae8b0b2c6eb46c95756cde6bcc7b6174157f501150778dfbfe69ef102992399a"} Jul 16 16:27:03.215282 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.215115 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:27:03.216280 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.216255 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wsdw" event={"ID":"b40a9b93-01a7-422f-b912-6b9c08a6d64e","Type":"ContainerStarted","Data":"b39861851c69a41ddf464f94f49954301634d0fdc724471cbc38fef0c37a398d"} Jul 16 16:27:03.216771 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.216748 2560 patch_prober.go:28] interesting pod/downloads-68c9cbb497-gwrlf container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" start-of-body= Jul 16 16:27:03.216846 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.216798 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-gwrlf" podUID="774b6eef-91d7-4267-8f74-f538e73313d1" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" Jul 16 16:27:03.316890 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.316779 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-gwrlf" podStartSLOduration=4.608252138 podStartE2EDuration="26.316760065s" totalImagesPullingTime="21.708507927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:26:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:27:03.234529951 +0000 UTC m=+99.042641965" watchObservedRunningTime="2026-07-16 16:27:03.316760065 +0000 UTC m=+99.124872079" Jul 16 16:27:03.317470 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:03.317443 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b4cdb78f7-jz5rc"] Jul 16 16:27:04.230845 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:04.229375 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-gwrlf" Jul 16 16:27:05.234848 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:05.234809 2560 generic.go:356] "Generic (PLEG): container finished" podID="b40a9b93-01a7-422f-b912-6b9c08a6d64e" containerID="5da0b357fc07378bed2d93482098ba08ccc189f6fa5462364b32049e856a3d06" exitCode=0 Jul 16 16:27:05.235290 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:05.234924 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wsdw" event={"ID":"b40a9b93-01a7-422f-b912-6b9c08a6d64e","Type":"ContainerDied","Data":"5da0b357fc07378bed2d93482098ba08ccc189f6fa5462364b32049e856a3d06"} Jul 16 16:27:06.077402 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:06.077365 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hzbgf" Jul 16 16:27:06.242319 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:06.242277 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wsdw" event={"ID":"b40a9b93-01a7-422f-b912-6b9c08a6d64e","Type":"ContainerStarted","Data":"67445fa44fbf920c54a18a84a528c2c01efdbc37712f1d3e54bc220ae59fe69b"} Jul 16 16:27:06.242790 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:06.242344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wsdw" event={"ID":"b40a9b93-01a7-422f-b912-6b9c08a6d64e","Type":"ContainerStarted","Data":"f747b566482c2663aff5cb1aff52bb54acc6946bbd29ec451b00e7bd82e7bd08"} Jul 16 16:27:10.397978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.397885 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7wsdw" podStartSLOduration=9.022896877 podStartE2EDuration="11.39786773s" totalImagesPullingTime="2.26755566s" totalInitContainerRuntime="107.415193ms" isStatefulPod=true podCreationTimestamp="2026-07-16 16:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:27:06.261989146 +0000 UTC m=+102.070101163" watchObservedRunningTime="2026-07-16 16:27:10.39786773 +0000 UTC m=+106.205979743" Jul 16 16:27:10.398593 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.398380 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75f896b47d-5zhlm"] Jul 16 16:27:10.495435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.495394 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75f896b47d-5zhlm"] Jul 16 16:27:10.495628 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.495526 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.499840 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.499816 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 16:27:10.500654 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.500630 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 16:27:10.501903 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.501882 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 16:27:10.502226 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.502207 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jx67x\"" Jul 16 16:27:10.502319 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.502257 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 16:27:10.502378 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.502257 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 16:27:10.506855 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.506832 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 16:27:10.591318 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-oauth-config\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.591513 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591362 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwcg6\" (UniqueName: \"kubernetes.io/projected/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-kube-api-access-rwcg6\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.591513 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-service-ca\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.591624 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-oauth-serving-cert\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.591624 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-config\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.591624 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-trusted-ca-bundle\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.591759 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.591639 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-serving-cert\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693098 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-service-ca\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693098 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-oauth-serving-cert\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693369 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-config\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693369 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-trusted-ca-bundle\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693369 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-serving-cert\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693369 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-oauth-config\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693369 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwcg6\" (UniqueName: \"kubernetes.io/projected/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-kube-api-access-rwcg6\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.693927 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693898 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-config\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.694030 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693939 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-service-ca\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.694030 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.693978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-oauth-serving-cert\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.694446 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.694424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-trusted-ca-bundle\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.696301 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.696279 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-oauth-config\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.697284 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.697258 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-serving-cert\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.717978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.717946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwcg6\" (UniqueName: \"kubernetes.io/projected/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-kube-api-access-rwcg6\") pod \"console-75f896b47d-5zhlm\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:10.807599 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:10.807560 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:11.021476 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:11.021377 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75f896b47d-5zhlm"] Jul 16 16:27:11.025439 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:27:11.025404 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7671d064_99f8_4de9_b1d0_1fb759f9b6d8.slice/crio-d45abfcc6c6e23a4d1bb646ad62be6bcd482eb678fbf7c08c784232c3aa4946e WatchSource:0}: Error finding container d45abfcc6c6e23a4d1bb646ad62be6bcd482eb678fbf7c08c784232c3aa4946e: Status 404 returned error can't find the container with id d45abfcc6c6e23a4d1bb646ad62be6bcd482eb678fbf7c08c784232c3aa4946e Jul 16 16:27:11.265785 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:11.265746 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f896b47d-5zhlm" event={"ID":"7671d064-99f8-4de9-b1d0-1fb759f9b6d8","Type":"ContainerStarted","Data":"d45abfcc6c6e23a4d1bb646ad62be6bcd482eb678fbf7c08c784232c3aa4946e"} Jul 16 16:27:16.293865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:16.293777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f896b47d-5zhlm" event={"ID":"7671d064-99f8-4de9-b1d0-1fb759f9b6d8","Type":"ContainerStarted","Data":"7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c"} Jul 16 16:27:16.313460 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:16.313405 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75f896b47d-5zhlm" podStartSLOduration=1.391975347 podStartE2EDuration="6.313388458s" totalImagesPullingTime="4.921413111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:27:16.312401667 +0000 UTC m=+112.120513682" watchObservedRunningTime="2026-07-16 16:27:16.313388458 +0000 UTC m=+112.121500472" Jul 16 16:27:20.807951 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:20.807914 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:20.808453 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:20.808002 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:20.812851 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:20.812830 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:21.314944 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:21.314904 2560 generic.go:356] "Generic (PLEG): container finished" podID="e03752d9-97fa-402a-aca0-a5cec5a740f6" containerID="371756a26573bbcfd32fa948a9a13e33b3fd2f2d08b3270c8d63e05b9b6b5143" exitCode=0 Jul 16 16:27:21.315168 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:21.314976 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" event={"ID":"e03752d9-97fa-402a-aca0-a5cec5a740f6","Type":"ContainerDied","Data":"371756a26573bbcfd32fa948a9a13e33b3fd2f2d08b3270c8d63e05b9b6b5143"} Jul 16 16:27:21.315513 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:21.315486 2560 scope.go:117] "RemoveContainer" containerID="371756a26573bbcfd32fa948a9a13e33b3fd2f2d08b3270c8d63e05b9b6b5143" Jul 16 16:27:21.319213 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:21.319194 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:27:22.320556 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:22.320520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-r2z8k" event={"ID":"e03752d9-97fa-402a-aca0-a5cec5a740f6","Type":"ContainerStarted","Data":"61ce405842aeee426c1f9ac448e0d0c2bb9963c96e3c9a59f242cca0a88b71da"} Jul 16 16:27:26.337777 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:26.337736 2560 generic.go:356] "Generic (PLEG): container finished" podID="bf714cea-2ff9-4200-ad09-6c3f7eba5ae4" containerID="7459763f175b9b37ed9bfb57a257a0db0c548e509e0395a701efa3398a7daeae" exitCode=0 Jul 16 16:27:26.338185 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:26.337819 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" event={"ID":"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4","Type":"ContainerDied","Data":"7459763f175b9b37ed9bfb57a257a0db0c548e509e0395a701efa3398a7daeae"} Jul 16 16:27:26.338227 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:26.338205 2560 scope.go:117] "RemoveContainer" containerID="7459763f175b9b37ed9bfb57a257a0db0c548e509e0395a701efa3398a7daeae" Jul 16 16:27:27.342947 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:27.342912 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2ks42" event={"ID":"bf714cea-2ff9-4200-ad09-6c3f7eba5ae4","Type":"ContainerStarted","Data":"2e38c8394c8f9a4b77730e2fa7c3bd19fcc5942f0a00e391a15ff95e7e198132"} Jul 16 16:27:28.351658 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.351605 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" podUID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" containerName="registry" containerID="cri-o://07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8" gracePeriod=30 Jul 16 16:27:28.636890 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.636853 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:27:28.644290 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644261 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-bound-sa-token\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644454 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644297 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpv5k\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-kube-api-access-mpv5k\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644454 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644346 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644567 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644455 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-certificates\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644567 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644504 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-installation-pull-secrets\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644567 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644536 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-image-registry-private-configuration\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644710 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644564 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-trusted-ca\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.644710 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644612 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-ca-trust-extracted\") pod \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\" (UID: \"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf\") " Jul 16 16:27:28.645329 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.644864 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:27:28.645329 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.645273 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:27:28.651216 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.651182 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:27:28.651353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.651208 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:27:28.651353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.651288 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:27:28.651353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.651309 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:27:28.651467 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.651392 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-kube-api-access-mpv5k" (OuterVolumeSpecName: "kube-api-access-mpv5k") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "kube-api-access-mpv5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:27:28.654227 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.654157 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" (UID: "58d2f5b0-84f5-4acb-82b0-05c2f1be4baf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:27:28.746155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746124 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746150 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-registry-certificates\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746159 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-installation-pull-secrets\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746384 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746169 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-image-registry-private-configuration\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746384 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746179 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-trusted-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746384 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746187 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-ca-trust-extracted\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746384 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746196 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-bound-sa-token\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:28.746384 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:28.746205 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpv5k\" (UniqueName: \"kubernetes.io/projected/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf-kube-api-access-mpv5k\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:27:29.352976 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.352943 2560 generic.go:356] "Generic (PLEG): container finished" podID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" containerID="07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8" exitCode=0 Jul 16 16:27:29.353561 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.352986 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" Jul 16 16:27:29.353561 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.353033 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" event={"ID":"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf","Type":"ContainerDied","Data":"07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8"} Jul 16 16:27:29.353561 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.353120 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4cdb78f7-jz5rc" event={"ID":"58d2f5b0-84f5-4acb-82b0-05c2f1be4baf","Type":"ContainerDied","Data":"716cfa62fab0954613802f8587ffb395375f9784f9b5b97545d5f07cc63c2392"} Jul 16 16:27:29.353561 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.353154 2560 scope.go:117] "RemoveContainer" containerID="07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8" Jul 16 16:27:29.361523 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.361504 2560 scope.go:117] "RemoveContainer" containerID="07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8" Jul 16 16:27:29.361806 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:27:29.361785 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8\": container with ID starting with 07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8 not found: ID does not exist" containerID="07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8" Jul 16 16:27:29.361854 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.361815 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8"} err="failed to get container status \"07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8\": rpc error: code = NotFound desc = could not find container \"07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8\": container with ID starting with 07b003cc0aea993d4dc967663826448d4a0e47be00cc99a0cb2f0c6b0b66a7e8 not found: ID does not exist" Jul 16 16:27:29.381630 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.381457 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b4cdb78f7-jz5rc"] Jul 16 16:27:29.382700 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:29.382678 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b4cdb78f7-jz5rc"] Jul 16 16:27:30.654618 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:30.654583 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" path="/var/lib/kubelet/pods/58d2f5b0-84f5-4acb-82b0-05c2f1be4baf/volumes" Jul 16 16:27:41.399837 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:41.399800 2560 generic.go:356] "Generic (PLEG): container finished" podID="6675174b-9998-44d2-a629-89c04b7951a1" containerID="fd438916d0f09429866f9c9b96707a45da81beb46fab23c736d5d327c0455a31" exitCode=0 Jul 16 16:27:41.400339 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:41.399850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" event={"ID":"6675174b-9998-44d2-a629-89c04b7951a1","Type":"ContainerDied","Data":"fd438916d0f09429866f9c9b96707a45da81beb46fab23c736d5d327c0455a31"} Jul 16 16:27:41.400339 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:41.400197 2560 scope.go:117] "RemoveContainer" containerID="fd438916d0f09429866f9c9b96707a45da81beb46fab23c736d5d327c0455a31" Jul 16 16:27:42.405695 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:27:42.405656 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dlqfr" event={"ID":"6675174b-9998-44d2-a629-89c04b7951a1","Type":"ContainerStarted","Data":"8072f77cd1c314ad59ebecf965b1e7c3ffc578487c8698c71273dcab530c756c"} Jul 16 16:28:25.572531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.572502 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58c8588fd5-m485w"] Jul 16 16:28:25.572983 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.572917 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" containerName="registry" Jul 16 16:28:25.572983 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.572937 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" containerName="registry" Jul 16 16:28:25.573130 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.573073 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d2f5b0-84f5-4acb-82b0-05c2f1be4baf" containerName="registry" Jul 16 16:28:25.577467 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.577444 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.589069 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.589032 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58c8588fd5-m485w"] Jul 16 16:28:25.608809 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.608785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-config\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.608965 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.608814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-trusted-ca-bundle\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.608965 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.608838 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tlhp\" (UniqueName: \"kubernetes.io/projected/ef0f702a-2d09-4729-a88c-878c4087ba8e-kube-api-access-9tlhp\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.608965 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.608914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-oauth-serving-cert\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.609198 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.608973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-service-ca\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.609198 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.609005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-serving-cert\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.609198 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.609105 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-oauth-config\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.709872 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.709837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-serving-cert\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.709896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-oauth-config\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.709947 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-config\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.709969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-trusted-ca-bundle\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.710016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tlhp\" (UniqueName: \"kubernetes.io/projected/ef0f702a-2d09-4729-a88c-878c4087ba8e-kube-api-access-9tlhp\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.710053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-oauth-serving-cert\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.710123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-service-ca\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710870 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.710843 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-config\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710966 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.710876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-service-ca\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.710966 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.710892 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-oauth-serving-cert\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.711203 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.711178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-trusted-ca-bundle\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.712489 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.712468 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-oauth-config\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.712659 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.712640 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-serving-cert\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.718763 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.718741 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tlhp\" (UniqueName: \"kubernetes.io/projected/ef0f702a-2d09-4729-a88c-878c4087ba8e-kube-api-access-9tlhp\") pod \"console-58c8588fd5-m485w\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:25.886275 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:25.886189 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:26.013735 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:26.013704 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58c8588fd5-m485w"] Jul 16 16:28:26.016723 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:28:26.016687 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef0f702a_2d09_4729_a88c_878c4087ba8e.slice/crio-7980d80627d5204d6ed9afce8e33508037fc3305d8b4cc5a558f8a0346fb5d11 WatchSource:0}: Error finding container 7980d80627d5204d6ed9afce8e33508037fc3305d8b4cc5a558f8a0346fb5d11: Status 404 returned error can't find the container with id 7980d80627d5204d6ed9afce8e33508037fc3305d8b4cc5a558f8a0346fb5d11 Jul 16 16:28:26.582911 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:26.582863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c8588fd5-m485w" event={"ID":"ef0f702a-2d09-4729-a88c-878c4087ba8e","Type":"ContainerStarted","Data":"ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e"} Jul 16 16:28:26.583372 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:26.582920 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c8588fd5-m485w" event={"ID":"ef0f702a-2d09-4729-a88c-878c4087ba8e","Type":"ContainerStarted","Data":"7980d80627d5204d6ed9afce8e33508037fc3305d8b4cc5a558f8a0346fb5d11"} Jul 16 16:28:26.602162 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:26.602111 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58c8588fd5-m485w" podStartSLOduration=1.6020943170000002 podStartE2EDuration="1.602094317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:28:26.601243564 +0000 UTC m=+182.409355578" watchObservedRunningTime="2026-07-16 16:28:26.602094317 +0000 UTC m=+182.410206327" Jul 16 16:28:35.886698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:35.886648 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:35.887239 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:35.886721 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:35.891647 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:35.891621 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:36.629163 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:36.629136 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:28:36.765620 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:28:36.765585 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75f896b47d-5zhlm"] Jul 16 16:29:01.788457 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:01.788399 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75f896b47d-5zhlm" podUID="7671d064-99f8-4de9-b1d0-1fb759f9b6d8" containerName="console" containerID="cri-o://7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c" gracePeriod=15 Jul 16 16:29:02.035666 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.035642 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:29:02.183008 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.182975 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-oauth-serving-cert\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183165 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183014 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-config\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183165 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183097 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwcg6\" (UniqueName: \"kubernetes.io/projected/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-kube-api-access-rwcg6\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183165 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183140 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-serving-cert\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183172 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-oauth-config\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183227 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-service-ca\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183259 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-trusted-ca-bundle\") pod \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\" (UID: \"7671d064-99f8-4de9-b1d0-1fb759f9b6d8\") " Jul 16 16:29:02.183486 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183460 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:29:02.183636 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183601 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-config" (OuterVolumeSpecName: "console-config") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:29:02.183750 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183681 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-service-ca" (OuterVolumeSpecName: "service-ca") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:29:02.183750 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.183725 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:29:02.185565 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.185542 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:29:02.185995 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.185977 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:29:02.186039 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.186012 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-kube-api-access-rwcg6" (OuterVolumeSpecName: "kube-api-access-rwcg6") pod "7671d064-99f8-4de9-b1d0-1fb759f9b6d8" (UID: "7671d064-99f8-4de9-b1d0-1fb759f9b6d8"). InnerVolumeSpecName "kube-api-access-rwcg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:29:02.284361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284307 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.284361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284353 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-oauth-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.284361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284363 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.284361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284372 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-trusted-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.284628 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284382 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-oauth-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.284628 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284391 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-console-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.284628 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.284399 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwcg6\" (UniqueName: \"kubernetes.io/projected/7671d064-99f8-4de9-b1d0-1fb759f9b6d8-kube-api-access-rwcg6\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:02.719272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.719236 2560 generic.go:356] "Generic (PLEG): container finished" podID="7671d064-99f8-4de9-b1d0-1fb759f9b6d8" containerID="7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c" exitCode=0 Jul 16 16:29:02.719429 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.719287 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f896b47d-5zhlm" Jul 16 16:29:02.719429 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.719302 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f896b47d-5zhlm" event={"ID":"7671d064-99f8-4de9-b1d0-1fb759f9b6d8","Type":"ContainerDied","Data":"7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c"} Jul 16 16:29:02.719429 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.719353 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f896b47d-5zhlm" event={"ID":"7671d064-99f8-4de9-b1d0-1fb759f9b6d8","Type":"ContainerDied","Data":"d45abfcc6c6e23a4d1bb646ad62be6bcd482eb678fbf7c08c784232c3aa4946e"} Jul 16 16:29:02.719429 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.719373 2560 scope.go:117] "RemoveContainer" containerID="7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c" Jul 16 16:29:02.728632 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.728614 2560 scope.go:117] "RemoveContainer" containerID="7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c" Jul 16 16:29:02.728901 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:29:02.728881 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c\": container with ID starting with 7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c not found: ID does not exist" containerID="7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c" Jul 16 16:29:02.728984 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.728912 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c"} err="failed to get container status \"7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c\": rpc error: code = NotFound desc = could not find container \"7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c\": container with ID starting with 7b7d2ec40baf6322061c3ee9b08734348f24271aedb38e8256cc3c9678f47f4c not found: ID does not exist" Jul 16 16:29:02.736494 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.736470 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75f896b47d-5zhlm"] Jul 16 16:29:02.739553 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:02.739530 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75f896b47d-5zhlm"] Jul 16 16:29:04.654609 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:04.654573 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7671d064-99f8-4de9-b1d0-1fb759f9b6d8" path="/var/lib/kubelet/pods/7671d064-99f8-4de9-b1d0-1fb759f9b6d8/volumes" Jul 16 16:29:14.757546 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.757510 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk"] Jul 16 16:29:14.757992 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.757781 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7671d064-99f8-4de9-b1d0-1fb759f9b6d8" containerName="console" Jul 16 16:29:14.757992 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.757792 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7671d064-99f8-4de9-b1d0-1fb759f9b6d8" containerName="console" Jul 16 16:29:14.757992 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.757853 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7671d064-99f8-4de9-b1d0-1fb759f9b6d8" containerName="console" Jul 16 16:29:14.760865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.760849 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.763276 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.763253 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wx4d6\"" Jul 16 16:29:14.763386 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.763257 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 16:29:14.763861 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.763845 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 16:29:14.774088 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.774052 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk"] Jul 16 16:29:14.894635 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.894595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.894820 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.894647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hc47\" (UniqueName: \"kubernetes.io/projected/0455bec4-101d-4b60-8a49-c8f9cc4c920d-kube-api-access-6hc47\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.894820 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.894710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.995729 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.995692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.995921 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.995742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hc47\" (UniqueName: \"kubernetes.io/projected/0455bec4-101d-4b60-8a49-c8f9cc4c920d-kube-api-access-6hc47\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.995921 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.995769 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.996106 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.996085 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:14.996194 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:14.996170 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:15.004304 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:15.004277 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hc47\" (UniqueName: \"kubernetes.io/projected/0455bec4-101d-4b60-8a49-c8f9cc4c920d-kube-api-access-6hc47\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:15.070303 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:15.070224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:15.204322 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:15.204290 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk"] Jul 16 16:29:15.207801 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:29:15.207776 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0455bec4_101d_4b60_8a49_c8f9cc4c920d.slice/crio-f373ca93b23b74c36fc42aaabfc34156f714712c085214f889586f2a756c3481 WatchSource:0}: Error finding container f373ca93b23b74c36fc42aaabfc34156f714712c085214f889586f2a756c3481: Status 404 returned error can't find the container with id f373ca93b23b74c36fc42aaabfc34156f714712c085214f889586f2a756c3481 Jul 16 16:29:15.772462 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:15.772426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" event={"ID":"0455bec4-101d-4b60-8a49-c8f9cc4c920d","Type":"ContainerStarted","Data":"f373ca93b23b74c36fc42aaabfc34156f714712c085214f889586f2a756c3481"} Jul 16 16:29:24.805961 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:24.805877 2560 generic.go:356] "Generic (PLEG): container finished" podID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerID="19dded142b887cd55b58ba46a1717d707d83ba7f39e7b977aca5654b60ea2037" exitCode=0 Jul 16 16:29:24.806320 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:24.805959 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" event={"ID":"0455bec4-101d-4b60-8a49-c8f9cc4c920d","Type":"ContainerDied","Data":"19dded142b887cd55b58ba46a1717d707d83ba7f39e7b977aca5654b60ea2037"} Jul 16 16:29:27.825792 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:27.825753 2560 generic.go:356] "Generic (PLEG): container finished" podID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerID="0c81d9a430dbc3b5a657c49dadf8c6a0563eb267b90f6ea8bb8b448907d90ad4" exitCode=0 Jul 16 16:29:27.826216 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:27.825840 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" event={"ID":"0455bec4-101d-4b60-8a49-c8f9cc4c920d","Type":"ContainerDied","Data":"0c81d9a430dbc3b5a657c49dadf8c6a0563eb267b90f6ea8bb8b448907d90ad4"} Jul 16 16:29:36.867308 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:36.867270 2560 generic.go:356] "Generic (PLEG): container finished" podID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerID="916eddc31628c9bbc38983ce064483a5f4a906c4e78fa05deabdc4b90584801a" exitCode=0 Jul 16 16:29:36.867725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:36.867328 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" event={"ID":"0455bec4-101d-4b60-8a49-c8f9cc4c920d","Type":"ContainerDied","Data":"916eddc31628c9bbc38983ce064483a5f4a906c4e78fa05deabdc4b90584801a"} Jul 16 16:29:37.997741 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:37.997716 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:38.093032 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.092997 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hc47\" (UniqueName: \"kubernetes.io/projected/0455bec4-101d-4b60-8a49-c8f9cc4c920d-kube-api-access-6hc47\") pod \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " Jul 16 16:29:38.093258 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.093081 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-bundle\") pod \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " Jul 16 16:29:38.093258 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.093102 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-util\") pod \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\" (UID: \"0455bec4-101d-4b60-8a49-c8f9cc4c920d\") " Jul 16 16:29:38.093768 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.093741 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-bundle" (OuterVolumeSpecName: "bundle") pod "0455bec4-101d-4b60-8a49-c8f9cc4c920d" (UID: "0455bec4-101d-4b60-8a49-c8f9cc4c920d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:29:38.095410 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.095389 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0455bec4-101d-4b60-8a49-c8f9cc4c920d-kube-api-access-6hc47" (OuterVolumeSpecName: "kube-api-access-6hc47") pod "0455bec4-101d-4b60-8a49-c8f9cc4c920d" (UID: "0455bec4-101d-4b60-8a49-c8f9cc4c920d"). InnerVolumeSpecName "kube-api-access-6hc47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:29:38.098860 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.098829 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-util" (OuterVolumeSpecName: "util") pod "0455bec4-101d-4b60-8a49-c8f9cc4c920d" (UID: "0455bec4-101d-4b60-8a49-c8f9cc4c920d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:29:38.194276 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.194238 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hc47\" (UniqueName: \"kubernetes.io/projected/0455bec4-101d-4b60-8a49-c8f9cc4c920d-kube-api-access-6hc47\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:38.194276 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.194273 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:38.194476 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.194287 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0455bec4-101d-4b60-8a49-c8f9cc4c920d-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:38.876676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.876643 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" Jul 16 16:29:38.876676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.876662 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bspvk" event={"ID":"0455bec4-101d-4b60-8a49-c8f9cc4c920d","Type":"ContainerDied","Data":"f373ca93b23b74c36fc42aaabfc34156f714712c085214f889586f2a756c3481"} Jul 16 16:29:38.876878 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:38.876716 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f373ca93b23b74c36fc42aaabfc34156f714712c085214f889586f2a756c3481" Jul 16 16:29:43.049282 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049248 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84"] Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049555 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="pull" Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049567 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="pull" Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049585 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="extract" Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049590 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="extract" Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049602 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="util" Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049610 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="util" Jul 16 16:29:43.049725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.049657 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0455bec4-101d-4b60-8a49-c8f9cc4c920d" containerName="extract" Jul 16 16:29:43.054347 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.054328 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.056842 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.056816 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 16:29:43.056965 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.056874 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:29:43.056965 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.056889 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4cjl2\"" Jul 16 16:29:43.071993 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.071960 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84"] Jul 16 16:29:43.133193 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.133143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhtxg\" (UniqueName: \"kubernetes.io/projected/e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2-kube-api-access-hhtxg\") pod \"cert-manager-operator-controller-manager-55c444bf86-9rc84\" (UID: \"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.133389 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.133225 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9rc84\" (UID: \"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.233961 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.233928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9rc84\" (UID: \"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.234172 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.233982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhtxg\" (UniqueName: \"kubernetes.io/projected/e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2-kube-api-access-hhtxg\") pod \"cert-manager-operator-controller-manager-55c444bf86-9rc84\" (UID: \"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.234444 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.234422 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9rc84\" (UID: \"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.244912 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.244878 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhtxg\" (UniqueName: \"kubernetes.io/projected/e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2-kube-api-access-hhtxg\") pod \"cert-manager-operator-controller-manager-55c444bf86-9rc84\" (UID: \"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.363370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.363293 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" Jul 16 16:29:43.517125 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.517083 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84"] Jul 16 16:29:43.521132 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:29:43.521098 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e9c1eb_b10e_4c37_b912_3aa13fc3daf2.slice/crio-db844fae5e9d84b4f9273a5c0784619db4b8fb37b73aa408d5569b6c24758406 WatchSource:0}: Error finding container db844fae5e9d84b4f9273a5c0784619db4b8fb37b73aa408d5569b6c24758406: Status 404 returned error can't find the container with id db844fae5e9d84b4f9273a5c0784619db4b8fb37b73aa408d5569b6c24758406 Jul 16 16:29:43.896627 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:43.896587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" event={"ID":"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2","Type":"ContainerStarted","Data":"db844fae5e9d84b4f9273a5c0784619db4b8fb37b73aa408d5569b6c24758406"} Jul 16 16:29:45.908871 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:45.908825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" event={"ID":"e3e9c1eb-b10e-4c37-b912-3aa13fc3daf2","Type":"ContainerStarted","Data":"dcd67251f95cd26e9852c3772e0b727aa2c6c0c5cf123935b087001d370e2004"} Jul 16 16:29:45.934209 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:45.934148 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9rc84" podStartSLOduration=1.187770055 podStartE2EDuration="2.93413151s" totalImagesPullingTime="1.746361455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:29:45.931117121 +0000 UTC m=+261.739229134" watchObservedRunningTime="2026-07-16 16:29:45.93413151 +0000 UTC m=+261.742243527" Jul 16 16:29:47.031557 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.031527 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn"] Jul 16 16:29:47.035139 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.035119 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.041573 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.041314 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wx4d6\"" Jul 16 16:29:47.047239 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.047218 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 16:29:47.047397 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.047377 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 16:29:47.068710 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.068682 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn"] Jul 16 16:29:47.167472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.167439 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.167644 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.167489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.167644 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.167531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62cgh\" (UniqueName: \"kubernetes.io/projected/6967b73b-d968-4b52-9361-42ea0c16afc1-kube-api-access-62cgh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.268250 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.268214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.268250 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.268254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62cgh\" (UniqueName: \"kubernetes.io/projected/6967b73b-d968-4b52-9361-42ea0c16afc1-kube-api-access-62cgh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.268509 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.268301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.268707 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.268685 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.268778 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.268690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.318971 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.318879 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62cgh\" (UniqueName: \"kubernetes.io/projected/6967b73b-d968-4b52-9361-42ea0c16afc1-kube-api-access-62cgh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.343870 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.343832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:47.511367 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.511332 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn"] Jul 16 16:29:47.516413 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:29:47.516382 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6967b73b_d968_4b52_9361_42ea0c16afc1.slice/crio-08d7ce932092bac4433eccd09d4e34c82ded6b757a6589d35cd516d09c43b67c WatchSource:0}: Error finding container 08d7ce932092bac4433eccd09d4e34c82ded6b757a6589d35cd516d09c43b67c: Status 404 returned error can't find the container with id 08d7ce932092bac4433eccd09d4e34c82ded6b757a6589d35cd516d09c43b67c Jul 16 16:29:47.917892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.917855 2560 generic.go:356] "Generic (PLEG): container finished" podID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerID="3b4ae03fbc0db9767d18821da3f1458d3c4f17faa11404a6ce027843764c5b9f" exitCode=0 Jul 16 16:29:47.918071 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.917907 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" event={"ID":"6967b73b-d968-4b52-9361-42ea0c16afc1","Type":"ContainerDied","Data":"3b4ae03fbc0db9767d18821da3f1458d3c4f17faa11404a6ce027843764c5b9f"} Jul 16 16:29:47.918071 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:47.917968 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" event={"ID":"6967b73b-d968-4b52-9361-42ea0c16afc1","Type":"ContainerStarted","Data":"08d7ce932092bac4433eccd09d4e34c82ded6b757a6589d35cd516d09c43b67c"} Jul 16 16:29:50.934225 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:50.934184 2560 generic.go:356] "Generic (PLEG): container finished" podID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerID="7314d10fc57f3c48f871dc033336ba31b7bbf53a7641ca89e7abb7a134fe5742" exitCode=0 Jul 16 16:29:50.934664 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:50.934236 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" event={"ID":"6967b73b-d968-4b52-9361-42ea0c16afc1","Type":"ContainerDied","Data":"7314d10fc57f3c48f871dc033336ba31b7bbf53a7641ca89e7abb7a134fe5742"} Jul 16 16:29:51.940605 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:51.940575 2560 generic.go:356] "Generic (PLEG): container finished" podID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerID="9fed0bffe5fea6a55bfcb64b8c5d0ea51b1ced1785f396e34b299f09c8d033e6" exitCode=0 Jul 16 16:29:51.940971 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:51.940653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" event={"ID":"6967b73b-d968-4b52-9361-42ea0c16afc1","Type":"ContainerDied","Data":"9fed0bffe5fea6a55bfcb64b8c5d0ea51b1ced1785f396e34b299f09c8d033e6"} Jul 16 16:29:53.078295 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.078269 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:53.217527 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.217432 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62cgh\" (UniqueName: \"kubernetes.io/projected/6967b73b-d968-4b52-9361-42ea0c16afc1-kube-api-access-62cgh\") pod \"6967b73b-d968-4b52-9361-42ea0c16afc1\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " Jul 16 16:29:53.217527 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.217473 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-bundle\") pod \"6967b73b-d968-4b52-9361-42ea0c16afc1\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " Jul 16 16:29:53.217527 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.217518 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-util\") pod \"6967b73b-d968-4b52-9361-42ea0c16afc1\" (UID: \"6967b73b-d968-4b52-9361-42ea0c16afc1\") " Jul 16 16:29:53.217865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.217841 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-bundle" (OuterVolumeSpecName: "bundle") pod "6967b73b-d968-4b52-9361-42ea0c16afc1" (UID: "6967b73b-d968-4b52-9361-42ea0c16afc1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:29:53.219729 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.219700 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6967b73b-d968-4b52-9361-42ea0c16afc1-kube-api-access-62cgh" (OuterVolumeSpecName: "kube-api-access-62cgh") pod "6967b73b-d968-4b52-9361-42ea0c16afc1" (UID: "6967b73b-d968-4b52-9361-42ea0c16afc1"). InnerVolumeSpecName "kube-api-access-62cgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:29:53.222141 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.222120 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-util" (OuterVolumeSpecName: "util") pod "6967b73b-d968-4b52-9361-42ea0c16afc1" (UID: "6967b73b-d968-4b52-9361-42ea0c16afc1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:29:53.318844 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.318812 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62cgh\" (UniqueName: \"kubernetes.io/projected/6967b73b-d968-4b52-9361-42ea0c16afc1-kube-api-access-62cgh\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:53.318844 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.318837 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:53.318844 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.318846 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6967b73b-d968-4b52-9361-42ea0c16afc1-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:29:53.951375 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.951349 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" Jul 16 16:29:53.951542 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.951347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fgbnpn" event={"ID":"6967b73b-d968-4b52-9361-42ea0c16afc1","Type":"ContainerDied","Data":"08d7ce932092bac4433eccd09d4e34c82ded6b757a6589d35cd516d09c43b67c"} Jul 16 16:29:53.951542 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:53.951469 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08d7ce932092bac4433eccd09d4e34c82ded6b757a6589d35cd516d09c43b67c" Jul 16 16:29:59.254138 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254107 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd"] Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254385 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="extract" Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254395 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="extract" Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254412 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="util" Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254417 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="util" Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254425 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="pull" Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254431 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="pull" Jul 16 16:29:59.254483 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.254474 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6967b73b-d968-4b52-9361-42ea0c16afc1" containerName="extract" Jul 16 16:29:59.258257 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.258240 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.263948 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.263924 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hddbt\"" Jul 16 16:29:59.264134 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.264105 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 16:29:59.266138 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.266118 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 16:29:59.295680 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.295648 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd"] Jul 16 16:29:59.371716 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.371677 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6ed089f-0c77-419d-83c1-49abb8202f92-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rq5qd\" (UID: \"b6ed089f-0c77-419d-83c1-49abb8202f92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.371886 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.371735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwrv7\" (UniqueName: \"kubernetes.io/projected/b6ed089f-0c77-419d-83c1-49abb8202f92-kube-api-access-wwrv7\") pod \"openshift-lws-operator-fd8ccff4c-rq5qd\" (UID: \"b6ed089f-0c77-419d-83c1-49abb8202f92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.472373 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.472321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6ed089f-0c77-419d-83c1-49abb8202f92-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rq5qd\" (UID: \"b6ed089f-0c77-419d-83c1-49abb8202f92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.472550 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.472399 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrv7\" (UniqueName: \"kubernetes.io/projected/b6ed089f-0c77-419d-83c1-49abb8202f92-kube-api-access-wwrv7\") pod \"openshift-lws-operator-fd8ccff4c-rq5qd\" (UID: \"b6ed089f-0c77-419d-83c1-49abb8202f92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.472776 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.472756 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6ed089f-0c77-419d-83c1-49abb8202f92-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rq5qd\" (UID: \"b6ed089f-0c77-419d-83c1-49abb8202f92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.483605 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.483573 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwrv7\" (UniqueName: \"kubernetes.io/projected/b6ed089f-0c77-419d-83c1-49abb8202f92-kube-api-access-wwrv7\") pod \"openshift-lws-operator-fd8ccff4c-rq5qd\" (UID: \"b6ed089f-0c77-419d-83c1-49abb8202f92\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.540192 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.540105 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r4n6f"] Jul 16 16:29:59.543917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.543899 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.549102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.549082 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 16:29:59.549229 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.549212 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 16:29:59.549287 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.549256 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nzcmm\"" Jul 16 16:29:59.563430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.563301 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r4n6f"] Jul 16 16:29:59.567147 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.567123 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" Jul 16 16:29:59.674410 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.674371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ntn\" (UniqueName: \"kubernetes.io/projected/e9ae8afa-b666-4475-958a-08a776af5ddf-kube-api-access-m6ntn\") pod \"cert-manager-55b4d954d5-r4n6f\" (UID: \"e9ae8afa-b666-4475-958a-08a776af5ddf\") " pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.674567 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.674434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9ae8afa-b666-4475-958a-08a776af5ddf-bound-sa-token\") pod \"cert-manager-55b4d954d5-r4n6f\" (UID: \"e9ae8afa-b666-4475-958a-08a776af5ddf\") " pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.720093 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.720048 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd"] Jul 16 16:29:59.723033 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:29:59.723010 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6ed089f_0c77_419d_83c1_49abb8202f92.slice/crio-a41d5536e0b1e7ce1759fbece81da86859a1091bce2117d7dfe200a14a938b65 WatchSource:0}: Error finding container a41d5536e0b1e7ce1759fbece81da86859a1091bce2117d7dfe200a14a938b65: Status 404 returned error can't find the container with id a41d5536e0b1e7ce1759fbece81da86859a1091bce2117d7dfe200a14a938b65 Jul 16 16:29:59.775020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.774983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ntn\" (UniqueName: \"kubernetes.io/projected/e9ae8afa-b666-4475-958a-08a776af5ddf-kube-api-access-m6ntn\") pod \"cert-manager-55b4d954d5-r4n6f\" (UID: \"e9ae8afa-b666-4475-958a-08a776af5ddf\") " pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.775213 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.775030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9ae8afa-b666-4475-958a-08a776af5ddf-bound-sa-token\") pod \"cert-manager-55b4d954d5-r4n6f\" (UID: \"e9ae8afa-b666-4475-958a-08a776af5ddf\") " pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.795599 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.795538 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9ae8afa-b666-4475-958a-08a776af5ddf-bound-sa-token\") pod \"cert-manager-55b4d954d5-r4n6f\" (UID: \"e9ae8afa-b666-4475-958a-08a776af5ddf\") " pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.798934 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.798908 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ntn\" (UniqueName: \"kubernetes.io/projected/e9ae8afa-b666-4475-958a-08a776af5ddf-kube-api-access-m6ntn\") pod \"cert-manager-55b4d954d5-r4n6f\" (UID: \"e9ae8afa-b666-4475-958a-08a776af5ddf\") " pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.852771 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.852737 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r4n6f" Jul 16 16:29:59.975968 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:29:59.975931 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" event={"ID":"b6ed089f-0c77-419d-83c1-49abb8202f92","Type":"ContainerStarted","Data":"a41d5536e0b1e7ce1759fbece81da86859a1091bce2117d7dfe200a14a938b65"} Jul 16 16:30:00.040912 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:00.040880 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r4n6f"] Jul 16 16:30:00.043457 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:30:00.043421 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9ae8afa_b666_4475_958a_08a776af5ddf.slice/crio-2638176d31f5801769b4a49d2d6aeb73ec8cef48c8aef0fdc518d9a814b9a8b3 WatchSource:0}: Error finding container 2638176d31f5801769b4a49d2d6aeb73ec8cef48c8aef0fdc518d9a814b9a8b3: Status 404 returned error can't find the container with id 2638176d31f5801769b4a49d2d6aeb73ec8cef48c8aef0fdc518d9a814b9a8b3 Jul 16 16:30:00.982053 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:00.982015 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r4n6f" event={"ID":"e9ae8afa-b666-4475-958a-08a776af5ddf","Type":"ContainerStarted","Data":"2638176d31f5801769b4a49d2d6aeb73ec8cef48c8aef0fdc518d9a814b9a8b3"} Jul 16 16:30:01.988922 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:01.988887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" event={"ID":"b6ed089f-0c77-419d-83c1-49abb8202f92","Type":"ContainerStarted","Data":"1bf9c7b06ed7232f5a678929f53059b465b0c6b0891a1a8a0b8a5cb335b83869"} Jul 16 16:30:02.019961 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:02.019905 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rq5qd" podStartSLOduration=1.456438966 podStartE2EDuration="3.019884199s" totalImagesPullingTime="1.563445233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:30:02.018702291 +0000 UTC m=+277.826814306" watchObservedRunningTime="2026-07-16 16:30:02.019884199 +0000 UTC m=+277.827996214" Jul 16 16:30:05.005146 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.005111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r4n6f" event={"ID":"e9ae8afa-b666-4475-958a-08a776af5ddf","Type":"ContainerStarted","Data":"0fa1f3bf27c0a345a6ac4b8962f63d0e881cf69cd66b66a6650d810069e4a76e"} Jul 16 16:30:05.013410 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.013382 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr"] Jul 16 16:30:05.016812 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.016791 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.020179 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.020151 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wx4d6\"" Jul 16 16:30:05.020371 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.020353 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 16:30:05.020454 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.020437 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 16:30:05.038961 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.038914 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-r4n6f" podStartSLOduration=1.771023452 podStartE2EDuration="6.038895947s" totalImagesPullingTime="4.267872495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:29:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:30:05.037051173 +0000 UTC m=+280.845163187" watchObservedRunningTime="2026-07-16 16:30:05.038895947 +0000 UTC m=+280.847007960" Jul 16 16:30:05.043569 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.043543 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr"] Jul 16 16:30:05.123561 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.123516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.123561 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.123562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txdf2\" (UniqueName: \"kubernetes.io/projected/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-kube-api-access-txdf2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.123783 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.123681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.224631 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.224594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txdf2\" (UniqueName: \"kubernetes.io/projected/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-kube-api-access-txdf2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.224799 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.224662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.224799 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.224729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.225047 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.225028 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.225127 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.225112 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.240767 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.240732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txdf2\" (UniqueName: \"kubernetes.io/projected/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-kube-api-access-txdf2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.325783 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.325692 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:05.470623 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:05.470572 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr"] Jul 16 16:30:05.473220 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:30:05.473196 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ee4ba2_2523_4a7e_a3e0_7ed2d01db52f.slice/crio-88f841d1f6ae033a8daa22575163228b2fb9ef4d1084d269f04ac0472eb7e059 WatchSource:0}: Error finding container 88f841d1f6ae033a8daa22575163228b2fb9ef4d1084d269f04ac0472eb7e059: Status 404 returned error can't find the container with id 88f841d1f6ae033a8daa22575163228b2fb9ef4d1084d269f04ac0472eb7e059 Jul 16 16:30:06.010873 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:06.010838 2560 generic.go:356] "Generic (PLEG): container finished" podID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerID="f892c058d3f4b12f6f68e9c60c5d2374afe8eb0826a34ae112bad8f5a48ca269" exitCode=0 Jul 16 16:30:06.011323 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:06.010927 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" event={"ID":"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f","Type":"ContainerDied","Data":"f892c058d3f4b12f6f68e9c60c5d2374afe8eb0826a34ae112bad8f5a48ca269"} Jul 16 16:30:06.011323 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:06.010974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" event={"ID":"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f","Type":"ContainerStarted","Data":"88f841d1f6ae033a8daa22575163228b2fb9ef4d1084d269f04ac0472eb7e059"} Jul 16 16:30:07.016866 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:07.016829 2560 generic.go:356] "Generic (PLEG): container finished" podID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerID="f25e018119d5ce833da001b7bdd5c38a9de08cdceda3bd94c0b8247bfbb3caf8" exitCode=0 Jul 16 16:30:07.017264 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:07.016909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" event={"ID":"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f","Type":"ContainerDied","Data":"f25e018119d5ce833da001b7bdd5c38a9de08cdceda3bd94c0b8247bfbb3caf8"} Jul 16 16:30:08.023866 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:08.023832 2560 generic.go:356] "Generic (PLEG): container finished" podID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerID="3aa24474bfe6c0d37c55b178b382e85baf23618288f2f54eccf053f03e7ab666" exitCode=0 Jul 16 16:30:08.024286 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:08.023932 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" event={"ID":"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f","Type":"ContainerDied","Data":"3aa24474bfe6c0d37c55b178b382e85baf23618288f2f54eccf053f03e7ab666"} Jul 16 16:30:09.161334 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.161307 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:09.257543 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.257510 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txdf2\" (UniqueName: \"kubernetes.io/projected/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-kube-api-access-txdf2\") pod \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " Jul 16 16:30:09.257543 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.257550 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-bundle\") pod \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " Jul 16 16:30:09.257789 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.257574 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-util\") pod \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\" (UID: \"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f\") " Jul 16 16:30:09.258322 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.258285 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-bundle" (OuterVolumeSpecName: "bundle") pod "33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" (UID: "33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:30:09.259953 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.259924 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-kube-api-access-txdf2" (OuterVolumeSpecName: "kube-api-access-txdf2") pod "33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" (UID: "33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f"). InnerVolumeSpecName "kube-api-access-txdf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:30:09.265372 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.265331 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-util" (OuterVolumeSpecName: "util") pod "33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" (UID: "33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:30:09.358800 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.358709 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txdf2\" (UniqueName: \"kubernetes.io/projected/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-kube-api-access-txdf2\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:30:09.358800 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.358740 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:30:09.358800 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:09.358753 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:30:10.034573 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:10.034543 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" Jul 16 16:30:10.034573 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:10.034565 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbx8tr" event={"ID":"33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f","Type":"ContainerDied","Data":"88f841d1f6ae033a8daa22575163228b2fb9ef4d1084d269f04ac0472eb7e059"} Jul 16 16:30:10.034856 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:10.034613 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88f841d1f6ae033a8daa22575163228b2fb9ef4d1084d269f04ac0472eb7e059" Jul 16 16:30:20.568199 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568160 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj"] Jul 16 16:30:20.568673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568526 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="util" Jul 16 16:30:20.568673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568544 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="util" Jul 16 16:30:20.568673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568584 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="extract" Jul 16 16:30:20.568673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568593 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="extract" Jul 16 16:30:20.568673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568606 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="pull" Jul 16 16:30:20.568673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568613 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="pull" Jul 16 16:30:20.568910 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.568684 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ee4ba2-2523-4a7e-a3e0-7ed2d01db52f" containerName="extract" Jul 16 16:30:20.572741 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.572724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.576552 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.576524 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 16:30:20.576552 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.576529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 16:30:20.577345 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.577329 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 16:30:20.577498 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.577481 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7fcqd\"" Jul 16 16:30:20.577894 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.577880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 16:30:20.593180 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.593146 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj"] Jul 16 16:30:20.750208 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.750168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.750395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.750246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-tmp\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.750395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.750263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdk5p\" (UniqueName: \"kubernetes.io/projected/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-kube-api-access-xdk5p\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.750395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.750287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-webhook-cert\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.750395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.750341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-manifests\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.850799 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.850697 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.850799 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.850751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-tmp\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.850799 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.850776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdk5p\" (UniqueName: \"kubernetes.io/projected/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-kube-api-access-xdk5p\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.851099 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.850800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-webhook-cert\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.851099 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.850865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-manifests\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.851340 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.851318 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-manifests\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.853277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.853247 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-tmp\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.853369 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.853324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.853469 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.853450 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-webhook-cert\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.861645 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.861625 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdk5p\" (UniqueName: \"kubernetes.io/projected/b25dc11b-50e7-49ec-8183-0bf7918b9ffb-kube-api-access-xdk5p\") pod \"opendatahub-operator-controller-manager-846bd4c774-bgllj\" (UID: \"b25dc11b-50e7-49ec-8183-0bf7918b9ffb\") " pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:20.884545 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:20.884506 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:21.056734 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:21.056697 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj"] Jul 16 16:30:21.060543 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:30:21.060518 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25dc11b_50e7_49ec_8183_0bf7918b9ffb.slice/crio-4fc17573eecec63c14f76e8b15db2878b03f83ba8cd11ec8d53c36ca05f4d84c WatchSource:0}: Error finding container 4fc17573eecec63c14f76e8b15db2878b03f83ba8cd11ec8d53c36ca05f4d84c: Status 404 returned error can't find the container with id 4fc17573eecec63c14f76e8b15db2878b03f83ba8cd11ec8d53c36ca05f4d84c Jul 16 16:30:21.088130 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:21.088089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" event={"ID":"b25dc11b-50e7-49ec-8183-0bf7918b9ffb","Type":"ContainerStarted","Data":"4fc17573eecec63c14f76e8b15db2878b03f83ba8cd11ec8d53c36ca05f4d84c"} Jul 16 16:30:24.537317 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:24.537284 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:30:24.538873 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:24.538854 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:30:24.548946 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:24.548929 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 16:30:25.110894 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:25.110859 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" event={"ID":"b25dc11b-50e7-49ec-8183-0bf7918b9ffb","Type":"ContainerStarted","Data":"eb2d6c6b9d90ece7c11c10394c23ef6c2b21af2a57bf5fcded8c77399be7be64"} Jul 16 16:30:26.116934 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:26.116897 2560 generic.go:356] "Generic (PLEG): container finished" podID="b25dc11b-50e7-49ec-8183-0bf7918b9ffb" containerID="eb2d6c6b9d90ece7c11c10394c23ef6c2b21af2a57bf5fcded8c77399be7be64" exitCode=0 Jul 16 16:30:26.116934 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:26.116936 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" event={"ID":"b25dc11b-50e7-49ec-8183-0bf7918b9ffb","Type":"ContainerDied","Data":"eb2d6c6b9d90ece7c11c10394c23ef6c2b21af2a57bf5fcded8c77399be7be64"} Jul 16 16:30:26.117435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:26.117419 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:30:29.132605 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:29.132574 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" event={"ID":"b25dc11b-50e7-49ec-8183-0bf7918b9ffb","Type":"ContainerStarted","Data":"9d4962e0a2cb799d144060dce7c244811bea0757f2dd3bb80329081998844275"} Jul 16 16:30:29.132975 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:29.132714 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:29.162620 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:29.162569 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" podStartSLOduration=2.61725052 podStartE2EDuration="9.162554052s" totalImagesPullingTime="5.469756441s" totalInitContainerRuntime="1.075547091s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:30:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:30:29.16050495 +0000 UTC m=+304.968616963" watchObservedRunningTime="2026-07-16 16:30:29.162554052 +0000 UTC m=+304.970666067" Jul 16 16:30:40.139810 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:40.139780 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-846bd4c774-bgllj" Jul 16 16:30:45.127004 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.126962 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6747f69598-6gkbs"] Jul 16 16:30:45.130231 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.130202 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.139613 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.139587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tls-certs\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.139711 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.139622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xffk\" (UniqueName: \"kubernetes.io/projected/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-kube-api-access-4xffk\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.139754 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.139711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tmp\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.154557 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:45.154517 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-6747f69598-6gkbs\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" podUID="c8930eed-7dda-4ce3-a11f-3e9cc46f62ce" pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.162032 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:45.161996 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-dockercfg-4fhvf\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4fhvf\"" type="*v1.Secret" Jul 16 16:30:45.162235 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:45.162211 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-tls\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" Jul 16 16:30:45.162542 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:45.162525 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 16 16:30:45.179699 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.179666 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6747f69598-6gkbs"] Jul 16 16:30:45.240745 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.240707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tmp\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.240924 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.240759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tls-certs\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.240924 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.240782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xffk\" (UniqueName: \"kubernetes.io/projected/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-kube-api-access-4xffk\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.243041 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.243023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tmp\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.259596 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.259571 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xffk\" (UniqueName: \"kubernetes.io/projected/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-kube-api-access-4xffk\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:45.486340 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.486304 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dfhzq"] Jul 16 16:30:45.490167 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.490148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:45.496090 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.496002 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 16 16:30:45.496248 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.496124 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2czpk\"" Jul 16 16:30:45.541260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.541215 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dfhzq"] Jul 16 16:30:45.542592 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.542563 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a768f94-dae6-4570-99ea-87b24a7c3dd4-cert\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:45.542722 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.542604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hhn4\" (UniqueName: \"kubernetes.io/projected/7a768f94-dae6-4570-99ea-87b24a7c3dd4-kube-api-access-7hhn4\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:45.643732 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.643693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a768f94-dae6-4570-99ea-87b24a7c3dd4-cert\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:45.643933 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.643742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hhn4\" (UniqueName: \"kubernetes.io/projected/7a768f94-dae6-4570-99ea-87b24a7c3dd4-kube-api-access-7hhn4\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:45.643933 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:45.643838 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 16 16:30:45.643933 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:45.643908 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a768f94-dae6-4570-99ea-87b24a7c3dd4-cert podName:7a768f94-dae6-4570-99ea-87b24a7c3dd4 nodeName:}" failed. No retries permitted until 2026-07-16 16:30:46.143891 +0000 UTC m=+321.952002991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a768f94-dae6-4570-99ea-87b24a7c3dd4-cert") pod "odh-model-controller-7998c67dc4-dfhzq" (UID: "7a768f94-dae6-4570-99ea-87b24a7c3dd4") : secret "odh-model-controller-webhook-cert" not found Jul 16 16:30:45.671573 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.671542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hhn4\" (UniqueName: \"kubernetes.io/projected/7a768f94-dae6-4570-99ea-87b24a7c3dd4-kube-api-access-7hhn4\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:45.977382 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:45.977347 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4fhvf\"" Jul 16 16:30:46.147642 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.147612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a768f94-dae6-4570-99ea-87b24a7c3dd4-cert\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:46.150183 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.150163 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a768f94-dae6-4570-99ea-87b24a7c3dd4-cert\") pod \"odh-model-controller-7998c67dc4-dfhzq\" (UID: \"7a768f94-dae6-4570-99ea-87b24a7c3dd4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:46.241281 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:46.241185 2560 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: failed to sync secret cache: timed out waiting for the condition Jul 16 16:30:46.241281 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:30:46.241264 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tls-certs podName:c8930eed-7dda-4ce3-a11f-3e9cc46f62ce nodeName:}" failed. No retries permitted until 2026-07-16 16:30:46.741246016 +0000 UTC m=+322.549358007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tls-certs") pod "kube-auth-proxy-6747f69598-6gkbs" (UID: "c8930eed-7dda-4ce3-a11f-3e9cc46f62ce") : failed to sync secret cache: timed out waiting for the condition Jul 16 16:30:46.378808 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.378775 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 16 16:30:46.400986 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.400956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:46.434258 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.434224 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 16:30:46.663937 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.663912 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dfhzq"] Jul 16 16:30:46.665035 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:30:46.665009 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a768f94_dae6_4570_99ea_87b24a7c3dd4.slice/crio-47b40f54044c1b02f358700ac8aa2122ddd5b6ea5ea008dfa1da87e94cc776e7 WatchSource:0}: Error finding container 47b40f54044c1b02f358700ac8aa2122ddd5b6ea5ea008dfa1da87e94cc776e7: Status 404 returned error can't find the container with id 47b40f54044c1b02f358700ac8aa2122ddd5b6ea5ea008dfa1da87e94cc776e7 Jul 16 16:30:46.752531 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.752485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tls-certs\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:46.755141 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.755119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8930eed-7dda-4ce3-a11f-3e9cc46f62ce-tls-certs\") pod \"kube-auth-proxy-6747f69598-6gkbs\" (UID: \"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce\") " pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:46.940228 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:46.940195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" Jul 16 16:30:47.155049 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:47.155024 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6747f69598-6gkbs"] Jul 16 16:30:47.156954 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:30:47.156932 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8930eed_7dda_4ce3_a11f_3e9cc46f62ce.slice/crio-3c12ec98f4db7338eb222b2f9c58650d888774e56307f00f0f9d5d4b50e8d47c WatchSource:0}: Error finding container 3c12ec98f4db7338eb222b2f9c58650d888774e56307f00f0f9d5d4b50e8d47c: Status 404 returned error can't find the container with id 3c12ec98f4db7338eb222b2f9c58650d888774e56307f00f0f9d5d4b50e8d47c Jul 16 16:30:47.218425 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:47.218335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" event={"ID":"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce","Type":"ContainerStarted","Data":"3c12ec98f4db7338eb222b2f9c58650d888774e56307f00f0f9d5d4b50e8d47c"} Jul 16 16:30:47.219396 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:47.219376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" event={"ID":"7a768f94-dae6-4570-99ea-87b24a7c3dd4","Type":"ContainerStarted","Data":"47b40f54044c1b02f358700ac8aa2122ddd5b6ea5ea008dfa1da87e94cc776e7"} Jul 16 16:30:50.237373 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.237269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" event={"ID":"c8930eed-7dda-4ce3-a11f-3e9cc46f62ce","Type":"ContainerStarted","Data":"957658390a18b51de284181224137e088352d7df048b048932ac71fcedea5f9c"} Jul 16 16:30:50.310430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.310355 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6747f69598-6gkbs" podStartSLOduration=2.535026689 podStartE2EDuration="5.310329304s" totalImagesPullingTime="2.775302615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:30:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:30:50.267442497 +0000 UTC m=+326.075554510" watchObservedRunningTime="2026-07-16 16:30:50.310329304 +0000 UTC m=+326.118441321" Jul 16 16:30:50.310631 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.310541 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9"] Jul 16 16:30:50.314321 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.314294 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.320747 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.320709 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 16:30:50.320882 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.320823 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 16:30:50.321186 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.321162 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cmt5n\"" Jul 16 16:30:50.321481 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.321462 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 16:30:50.329968 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.329932 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9"] Jul 16 16:30:50.389145 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.389106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2774729f-db7d-491d-8a3c-2d0e265d9efa-cert\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.389342 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.389215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtvrk\" (UniqueName: \"kubernetes.io/projected/2774729f-db7d-491d-8a3c-2d0e265d9efa-kube-api-access-vtvrk\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.389342 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.389256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2774729f-db7d-491d-8a3c-2d0e265d9efa-metrics-cert\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.389342 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.389288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2774729f-db7d-491d-8a3c-2d0e265d9efa-manager-config\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.490548 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.490449 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2774729f-db7d-491d-8a3c-2d0e265d9efa-cert\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.490548 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.490517 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtvrk\" (UniqueName: \"kubernetes.io/projected/2774729f-db7d-491d-8a3c-2d0e265d9efa-kube-api-access-vtvrk\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.491306 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.490553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2774729f-db7d-491d-8a3c-2d0e265d9efa-metrics-cert\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.491306 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.490580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2774729f-db7d-491d-8a3c-2d0e265d9efa-manager-config\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.491666 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.491616 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2774729f-db7d-491d-8a3c-2d0e265d9efa-manager-config\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.494225 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.494194 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2774729f-db7d-491d-8a3c-2d0e265d9efa-cert\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.494866 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.494841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2774729f-db7d-491d-8a3c-2d0e265d9efa-metrics-cert\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.500772 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.500721 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtvrk\" (UniqueName: \"kubernetes.io/projected/2774729f-db7d-491d-8a3c-2d0e265d9efa-kube-api-access-vtvrk\") pod \"lws-controller-manager-6697dd89c8-gwsj9\" (UID: \"2774729f-db7d-491d-8a3c-2d0e265d9efa\") " pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.627820 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.627782 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:50.803073 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:50.803023 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9"] Jul 16 16:30:50.806046 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:30:50.806011 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2774729f_db7d_491d_8a3c_2d0e265d9efa.slice/crio-12d1def226c62169f3fe359e0da2ece144a97a64433caf6dde1a87354b9fae00 WatchSource:0}: Error finding container 12d1def226c62169f3fe359e0da2ece144a97a64433caf6dde1a87354b9fae00: Status 404 returned error can't find the container with id 12d1def226c62169f3fe359e0da2ece144a97a64433caf6dde1a87354b9fae00 Jul 16 16:30:51.244385 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:51.244346 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" event={"ID":"2774729f-db7d-491d-8a3c-2d0e265d9efa","Type":"ContainerStarted","Data":"12d1def226c62169f3fe359e0da2ece144a97a64433caf6dde1a87354b9fae00"} Jul 16 16:30:53.255576 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:53.255539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" event={"ID":"7a768f94-dae6-4570-99ea-87b24a7c3dd4","Type":"ContainerStarted","Data":"e5c05bfddb9b16e089ff76e4d6cb9801d83e6b224e86a8fb8627c1e6b8008a88"} Jul 16 16:30:53.256147 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:53.255679 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:30:53.287631 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:53.287564 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" podStartSLOduration=2.702257355 podStartE2EDuration="8.287548248s" totalImagesPullingTime="5.585290893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:30:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:30:53.285957503 +0000 UTC m=+329.094069520" watchObservedRunningTime="2026-07-16 16:30:53.287548248 +0000 UTC m=+329.095660263" Jul 16 16:30:55.267012 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:55.266912 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" event={"ID":"2774729f-db7d-491d-8a3c-2d0e265d9efa","Type":"ContainerStarted","Data":"200e414f0a8595b6c5fd54b8fd1b04a54bef6919bb74be1dd5f65d742e2c6cf5"} Jul 16 16:30:55.267420 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:55.267081 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:30:55.305116 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:30:55.305050 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" podStartSLOduration=1.175263384 podStartE2EDuration="5.305033767s" totalImagesPullingTime="4.129770383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:30:55.303139867 +0000 UTC m=+331.111251892" watchObservedRunningTime="2026-07-16 16:30:55.305033767 +0000 UTC m=+331.113145781" Jul 16 16:31:04.262420 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:31:04.262386 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dfhzq" Jul 16 16:31:06.274963 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:31:06.274919 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6697dd89c8-gwsj9" Jul 16 16:32:01.138633 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.138596 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q"] Jul 16 16:32:01.142201 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.142183 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.144913 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.144892 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6hjbv\"" Jul 16 16:32:01.145052 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.144900 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 16:32:01.145052 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.144954 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 16:32:01.151789 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.151767 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q"] Jul 16 16:32:01.199318 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.199284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l94h\" (UniqueName: \"kubernetes.io/projected/b617f699-e76c-4314-abc3-fecd3bc20468-kube-api-access-4l94h\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.199318 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.199326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.199530 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.199413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.300245 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.300202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l94h\" (UniqueName: \"kubernetes.io/projected/b617f699-e76c-4314-abc3-fecd3bc20468-kube-api-access-4l94h\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.300451 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.300255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.300451 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.300321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.300723 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.300702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.300793 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.300760 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.309246 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.309215 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l94h\" (UniqueName: \"kubernetes.io/projected/b617f699-e76c-4314-abc3-fecd3bc20468-kube-api-access-4l94h\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.452597 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.452551 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:01.592807 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.592775 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q"] Jul 16 16:32:01.595376 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:32:01.595347 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb617f699_e76c_4314_abc3_fecd3bc20468.slice/crio-681a2097d78bff10392619ac9ba57b9103c13125e362a7bb9e427f87f7403dc8 WatchSource:0}: Error finding container 681a2097d78bff10392619ac9ba57b9103c13125e362a7bb9e427f87f7403dc8: Status 404 returned error can't find the container with id 681a2097d78bff10392619ac9ba57b9103c13125e362a7bb9e427f87f7403dc8 Jul 16 16:32:01.933592 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.933557 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h"] Jul 16 16:32:01.937159 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.937138 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:01.946468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:01.946440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h"] Jul 16 16:32:02.008343 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.008303 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.008503 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.008375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq2s4\" (UniqueName: \"kubernetes.io/projected/d9784c48-00dc-4a0e-a9da-43cb4886c638-kube-api-access-cq2s4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.008503 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.008420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.109443 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.109407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.109619 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.109497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.109619 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.109526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq2s4\" (UniqueName: \"kubernetes.io/projected/d9784c48-00dc-4a0e-a9da-43cb4886c638-kube-api-access-cq2s4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.109835 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.109815 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.109893 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.109876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.119962 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.119930 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq2s4\" (UniqueName: \"kubernetes.io/projected/d9784c48-00dc-4a0e-a9da-43cb4886c638-kube-api-access-cq2s4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.247436 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.247353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:02.464863 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.464831 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h"] Jul 16 16:32:02.479240 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:32:02.479202 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9784c48_00dc_4a0e_a9da_43cb4886c638.slice/crio-e6525880ca28e9142ee8b2cc726363d53463596887371867881d05ce91aae5f1 WatchSource:0}: Error finding container e6525880ca28e9142ee8b2cc726363d53463596887371867881d05ce91aae5f1: Status 404 returned error can't find the container with id e6525880ca28e9142ee8b2cc726363d53463596887371867881d05ce91aae5f1 Jul 16 16:32:02.592399 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.592368 2560 generic.go:356] "Generic (PLEG): container finished" podID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerID="7673225927e5005485bc157fa71d0d1f37b0d492db875ee5de103cfc5a70dbe1" exitCode=0 Jul 16 16:32:02.592516 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.592447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" event={"ID":"d9784c48-00dc-4a0e-a9da-43cb4886c638","Type":"ContainerDied","Data":"7673225927e5005485bc157fa71d0d1f37b0d492db875ee5de103cfc5a70dbe1"} Jul 16 16:32:02.592516 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.592506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" event={"ID":"d9784c48-00dc-4a0e-a9da-43cb4886c638","Type":"ContainerStarted","Data":"e6525880ca28e9142ee8b2cc726363d53463596887371867881d05ce91aae5f1"} Jul 16 16:32:02.594003 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.593979 2560 generic.go:356] "Generic (PLEG): container finished" podID="b617f699-e76c-4314-abc3-fecd3bc20468" containerID="99bf17da985c547cb39064b35d4ba3d7312ca54e4f807f6ab14dd2359bb7b026" exitCode=0 Jul 16 16:32:02.594109 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.594048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" event={"ID":"b617f699-e76c-4314-abc3-fecd3bc20468","Type":"ContainerDied","Data":"99bf17da985c547cb39064b35d4ba3d7312ca54e4f807f6ab14dd2359bb7b026"} Jul 16 16:32:02.594109 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.594097 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" event={"ID":"b617f699-e76c-4314-abc3-fecd3bc20468","Type":"ContainerStarted","Data":"681a2097d78bff10392619ac9ba57b9103c13125e362a7bb9e427f87f7403dc8"} Jul 16 16:32:02.792949 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.792863 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z"] Jul 16 16:32:02.796457 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.796433 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:02.819306 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.819277 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z"] Jul 16 16:32:02.917219 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.917183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:02.917404 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.917259 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:02.917404 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:02.917302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95q7\" (UniqueName: \"kubernetes.io/projected/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-kube-api-access-c95q7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.018418 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.018365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95q7\" (UniqueName: \"kubernetes.io/projected/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-kube-api-access-c95q7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.018631 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.018454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.018631 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.018513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.018886 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.018860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.018961 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.018931 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.032276 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.032226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95q7\" (UniqueName: \"kubernetes.io/projected/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-kube-api-access-c95q7\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.106231 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.106144 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:03.156468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.156426 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z"] Jul 16 16:32:03.160280 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.160239 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.178894 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.178865 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z"] Jul 16 16:32:03.221111 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.221045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.221271 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.221139 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.221271 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.221215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrfv2\" (UniqueName: \"kubernetes.io/projected/92f49632-cded-4ff8-b203-f9b404405bac-kube-api-access-jrfv2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.268270 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.268245 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z"] Jul 16 16:32:03.269565 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:32:03.269538 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8fe052_d9c9_4c37_b6b7_f5f2252d6673.slice/crio-6995632d58eeebac1b393441fd73f56fec522593a8aaad43c19661db68553077 WatchSource:0}: Error finding container 6995632d58eeebac1b393441fd73f56fec522593a8aaad43c19661db68553077: Status 404 returned error can't find the container with id 6995632d58eeebac1b393441fd73f56fec522593a8aaad43c19661db68553077 Jul 16 16:32:03.322188 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.322160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrfv2\" (UniqueName: \"kubernetes.io/projected/92f49632-cded-4ff8-b203-f9b404405bac-kube-api-access-jrfv2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.322310 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.322242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.322310 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.322282 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.322610 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.322590 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.322682 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.322656 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.336127 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.336104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrfv2\" (UniqueName: \"kubernetes.io/projected/92f49632-cded-4ff8-b203-f9b404405bac-kube-api-access-jrfv2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.475445 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.475409 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:03.602779 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.602751 2560 generic.go:356] "Generic (PLEG): container finished" podID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerID="8674865dc34c5c4048747fec71811ea5a76ac478611815c0e91e88bbcea3941b" exitCode=0 Jul 16 16:32:03.602922 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.602824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" event={"ID":"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673","Type":"ContainerDied","Data":"8674865dc34c5c4048747fec71811ea5a76ac478611815c0e91e88bbcea3941b"} Jul 16 16:32:03.602922 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.602872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" event={"ID":"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673","Type":"ContainerStarted","Data":"6995632d58eeebac1b393441fd73f56fec522593a8aaad43c19661db68553077"} Jul 16 16:32:03.654852 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:03.654828 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z"] Jul 16 16:32:03.687011 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:32:03.686969 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f49632_cded_4ff8_b203_f9b404405bac.slice/crio-ce9f051903338bf39daa6d9cb15cb446e6e8ec6220f439c6ed89c162335ede49 WatchSource:0}: Error finding container ce9f051903338bf39daa6d9cb15cb446e6e8ec6220f439c6ed89c162335ede49: Status 404 returned error can't find the container with id ce9f051903338bf39daa6d9cb15cb446e6e8ec6220f439c6ed89c162335ede49 Jul 16 16:32:04.610047 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.609951 2560 generic.go:356] "Generic (PLEG): container finished" podID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerID="85e786a0670c949898a78b5c21b470214c74b6e27c8fbc8bc8fcb2c54e0a8757" exitCode=0 Jul 16 16:32:04.610491 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.610038 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" event={"ID":"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673","Type":"ContainerDied","Data":"85e786a0670c949898a78b5c21b470214c74b6e27c8fbc8bc8fcb2c54e0a8757"} Jul 16 16:32:04.611630 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.611610 2560 generic.go:356] "Generic (PLEG): container finished" podID="92f49632-cded-4ff8-b203-f9b404405bac" containerID="f127bb3a09bcb670419c3a0e8abc4cb7f57fbfaa0d91efbe5a04788dcb4e2a4a" exitCode=0 Jul 16 16:32:04.611699 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.611678 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" event={"ID":"92f49632-cded-4ff8-b203-f9b404405bac","Type":"ContainerDied","Data":"f127bb3a09bcb670419c3a0e8abc4cb7f57fbfaa0d91efbe5a04788dcb4e2a4a"} Jul 16 16:32:04.611741 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.611727 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" event={"ID":"92f49632-cded-4ff8-b203-f9b404405bac","Type":"ContainerStarted","Data":"ce9f051903338bf39daa6d9cb15cb446e6e8ec6220f439c6ed89c162335ede49"} Jul 16 16:32:04.613594 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.613572 2560 generic.go:356] "Generic (PLEG): container finished" podID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerID="c94bde2bd8fc9cd68d857650e0810cc790dde31ab30081e515de9799235c7675" exitCode=0 Jul 16 16:32:04.613701 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:04.613630 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" event={"ID":"d9784c48-00dc-4a0e-a9da-43cb4886c638","Type":"ContainerDied","Data":"c94bde2bd8fc9cd68d857650e0810cc790dde31ab30081e515de9799235c7675"} Jul 16 16:32:05.623943 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:05.623844 2560 generic.go:356] "Generic (PLEG): container finished" podID="92f49632-cded-4ff8-b203-f9b404405bac" containerID="0b76a56ef10a78049bf6184efc4d7c75573bb47fadfc84b85a80665ed6b610d5" exitCode=0 Jul 16 16:32:05.624418 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:05.623945 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" event={"ID":"92f49632-cded-4ff8-b203-f9b404405bac","Type":"ContainerDied","Data":"0b76a56ef10a78049bf6184efc4d7c75573bb47fadfc84b85a80665ed6b610d5"} Jul 16 16:32:05.626159 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:05.626135 2560 generic.go:356] "Generic (PLEG): container finished" podID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerID="532fc4c5689e6eaf25f775a5397ac1173eabba1d2f0168d9646d11856f78bf22" exitCode=0 Jul 16 16:32:05.626280 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:05.626218 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" event={"ID":"d9784c48-00dc-4a0e-a9da-43cb4886c638","Type":"ContainerDied","Data":"532fc4c5689e6eaf25f775a5397ac1173eabba1d2f0168d9646d11856f78bf22"} Jul 16 16:32:05.628194 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:05.628175 2560 generic.go:356] "Generic (PLEG): container finished" podID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerID="f3c680a692bbafa4f2a15a4ffbcaa149d6aa2a1d944e6144349c78f1cc31186a" exitCode=0 Jul 16 16:32:05.628264 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:05.628210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" event={"ID":"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673","Type":"ContainerDied","Data":"f3c680a692bbafa4f2a15a4ffbcaa149d6aa2a1d944e6144349c78f1cc31186a"} Jul 16 16:32:06.636297 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.636259 2560 generic.go:356] "Generic (PLEG): container finished" podID="b617f699-e76c-4314-abc3-fecd3bc20468" containerID="5b9d77a71fb52a76b0d6f9bcb6c442dce20133d61c263cf272e7654a3d725a75" exitCode=0 Jul 16 16:32:06.636747 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.636346 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" event={"ID":"b617f699-e76c-4314-abc3-fecd3bc20468","Type":"ContainerDied","Data":"5b9d77a71fb52a76b0d6f9bcb6c442dce20133d61c263cf272e7654a3d725a75"} Jul 16 16:32:06.638535 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.638513 2560 generic.go:356] "Generic (PLEG): container finished" podID="92f49632-cded-4ff8-b203-f9b404405bac" containerID="ca3c933b1eb5e57c762573639002a84540bf6ea093d5991d25bd2f8847dc2d79" exitCode=0 Jul 16 16:32:06.638636 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.638599 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" event={"ID":"92f49632-cded-4ff8-b203-f9b404405bac","Type":"ContainerDied","Data":"ca3c933b1eb5e57c762573639002a84540bf6ea093d5991d25bd2f8847dc2d79"} Jul 16 16:32:06.789667 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.789641 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:06.845301 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.845272 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:06.853963 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.853938 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq2s4\" (UniqueName: \"kubernetes.io/projected/d9784c48-00dc-4a0e-a9da-43cb4886c638-kube-api-access-cq2s4\") pod \"d9784c48-00dc-4a0e-a9da-43cb4886c638\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " Jul 16 16:32:06.854136 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.853970 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-util\") pod \"d9784c48-00dc-4a0e-a9da-43cb4886c638\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " Jul 16 16:32:06.854136 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.853993 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-bundle\") pod \"d9784c48-00dc-4a0e-a9da-43cb4886c638\" (UID: \"d9784c48-00dc-4a0e-a9da-43cb4886c638\") " Jul 16 16:32:06.854725 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.854693 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-bundle" (OuterVolumeSpecName: "bundle") pod "d9784c48-00dc-4a0e-a9da-43cb4886c638" (UID: "d9784c48-00dc-4a0e-a9da-43cb4886c638"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:06.856337 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.856312 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9784c48-00dc-4a0e-a9da-43cb4886c638-kube-api-access-cq2s4" (OuterVolumeSpecName: "kube-api-access-cq2s4") pod "d9784c48-00dc-4a0e-a9da-43cb4886c638" (UID: "d9784c48-00dc-4a0e-a9da-43cb4886c638"). InnerVolumeSpecName "kube-api-access-cq2s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:32:06.859102 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.859045 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-util" (OuterVolumeSpecName: "util") pod "d9784c48-00dc-4a0e-a9da-43cb4886c638" (UID: "d9784c48-00dc-4a0e-a9da-43cb4886c638"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:06.955172 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955137 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c95q7\" (UniqueName: \"kubernetes.io/projected/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-kube-api-access-c95q7\") pod \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " Jul 16 16:32:06.955358 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955238 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-util\") pod \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " Jul 16 16:32:06.955358 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955336 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-bundle\") pod \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\" (UID: \"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673\") " Jul 16 16:32:06.955601 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955569 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq2s4\" (UniqueName: \"kubernetes.io/projected/d9784c48-00dc-4a0e-a9da-43cb4886c638-kube-api-access-cq2s4\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:06.955601 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955588 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:06.955601 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955598 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9784c48-00dc-4a0e-a9da-43cb4886c638-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:06.955841 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.955792 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-bundle" (OuterVolumeSpecName: "bundle") pod "7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" (UID: "7e8fe052-d9c9-4c37-b6b7-f5f2252d6673"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:06.957423 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.957401 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-kube-api-access-c95q7" (OuterVolumeSpecName: "kube-api-access-c95q7") pod "7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" (UID: "7e8fe052-d9c9-4c37-b6b7-f5f2252d6673"). InnerVolumeSpecName "kube-api-access-c95q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:32:06.960862 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:06.960838 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-util" (OuterVolumeSpecName: "util") pod "7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" (UID: "7e8fe052-d9c9-4c37-b6b7-f5f2252d6673"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:07.056353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.056261 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c95q7\" (UniqueName: \"kubernetes.io/projected/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-kube-api-access-c95q7\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:07.056353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.056291 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:07.056353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.056300 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e8fe052-d9c9-4c37-b6b7-f5f2252d6673-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:07.645539 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.645508 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" Jul 16 16:32:07.646008 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.645506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h" event={"ID":"d9784c48-00dc-4a0e-a9da-43cb4886c638","Type":"ContainerDied","Data":"e6525880ca28e9142ee8b2cc726363d53463596887371867881d05ce91aae5f1"} Jul 16 16:32:07.646008 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.645633 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6525880ca28e9142ee8b2cc726363d53463596887371867881d05ce91aae5f1" Jul 16 16:32:07.647471 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.647411 2560 generic.go:356] "Generic (PLEG): container finished" podID="b617f699-e76c-4314-abc3-fecd3bc20468" containerID="d6987a01a6dce5d42fad3616d6e5ce461030846dc44fcc09398200a1e6aed538" exitCode=0 Jul 16 16:32:07.647591 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.647460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" event={"ID":"b617f699-e76c-4314-abc3-fecd3bc20468","Type":"ContainerDied","Data":"d6987a01a6dce5d42fad3616d6e5ce461030846dc44fcc09398200a1e6aed538"} Jul 16 16:32:07.649264 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.649246 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" Jul 16 16:32:07.649379 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.649272 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z" event={"ID":"7e8fe052-d9c9-4c37-b6b7-f5f2252d6673","Type":"ContainerDied","Data":"6995632d58eeebac1b393441fd73f56fec522593a8aaad43c19661db68553077"} Jul 16 16:32:07.649379 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.649308 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6995632d58eeebac1b393441fd73f56fec522593a8aaad43c19661db68553077" Jul 16 16:32:07.796240 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.796214 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:07.964748 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.964718 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrfv2\" (UniqueName: \"kubernetes.io/projected/92f49632-cded-4ff8-b203-f9b404405bac-kube-api-access-jrfv2\") pod \"92f49632-cded-4ff8-b203-f9b404405bac\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " Jul 16 16:32:07.964943 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.964757 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-bundle\") pod \"92f49632-cded-4ff8-b203-f9b404405bac\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " Jul 16 16:32:07.964943 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.964786 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-util\") pod \"92f49632-cded-4ff8-b203-f9b404405bac\" (UID: \"92f49632-cded-4ff8-b203-f9b404405bac\") " Jul 16 16:32:07.965273 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.965251 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-bundle" (OuterVolumeSpecName: "bundle") pod "92f49632-cded-4ff8-b203-f9b404405bac" (UID: "92f49632-cded-4ff8-b203-f9b404405bac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:07.966998 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.966975 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92f49632-cded-4ff8-b203-f9b404405bac-kube-api-access-jrfv2" (OuterVolumeSpecName: "kube-api-access-jrfv2") pod "92f49632-cded-4ff8-b203-f9b404405bac" (UID: "92f49632-cded-4ff8-b203-f9b404405bac"). InnerVolumeSpecName "kube-api-access-jrfv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:32:07.973068 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:07.973007 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-util" (OuterVolumeSpecName: "util") pod "92f49632-cded-4ff8-b203-f9b404405bac" (UID: "92f49632-cded-4ff8-b203-f9b404405bac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:08.065884 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.065798 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrfv2\" (UniqueName: \"kubernetes.io/projected/92f49632-cded-4ff8-b203-f9b404405bac-kube-api-access-jrfv2\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:08.065884 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.065829 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:08.065884 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.065838 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/92f49632-cded-4ff8-b203-f9b404405bac-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:08.663779 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.663748 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" Jul 16 16:32:08.663779 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.663756 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z" event={"ID":"92f49632-cded-4ff8-b203-f9b404405bac","Type":"ContainerDied","Data":"ce9f051903338bf39daa6d9cb15cb446e6e8ec6220f439c6ed89c162335ede49"} Jul 16 16:32:08.664209 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.663808 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce9f051903338bf39daa6d9cb15cb446e6e8ec6220f439c6ed89c162335ede49" Jul 16 16:32:08.794589 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.794565 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:08.873510 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.873474 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l94h\" (UniqueName: \"kubernetes.io/projected/b617f699-e76c-4314-abc3-fecd3bc20468-kube-api-access-4l94h\") pod \"b617f699-e76c-4314-abc3-fecd3bc20468\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " Jul 16 16:32:08.873704 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.873586 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-bundle\") pod \"b617f699-e76c-4314-abc3-fecd3bc20468\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " Jul 16 16:32:08.873704 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.873631 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-util\") pod \"b617f699-e76c-4314-abc3-fecd3bc20468\" (UID: \"b617f699-e76c-4314-abc3-fecd3bc20468\") " Jul 16 16:32:08.873993 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.873970 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-bundle" (OuterVolumeSpecName: "bundle") pod "b617f699-e76c-4314-abc3-fecd3bc20468" (UID: "b617f699-e76c-4314-abc3-fecd3bc20468"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:08.875847 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.875824 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b617f699-e76c-4314-abc3-fecd3bc20468-kube-api-access-4l94h" (OuterVolumeSpecName: "kube-api-access-4l94h") pod "b617f699-e76c-4314-abc3-fecd3bc20468" (UID: "b617f699-e76c-4314-abc3-fecd3bc20468"). InnerVolumeSpecName "kube-api-access-4l94h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:32:08.878479 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.878456 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-util" (OuterVolumeSpecName: "util") pod "b617f699-e76c-4314-abc3-fecd3bc20468" (UID: "b617f699-e76c-4314-abc3-fecd3bc20468"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:32:08.974244 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.974201 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-util\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:08.974244 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.974237 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l94h\" (UniqueName: \"kubernetes.io/projected/b617f699-e76c-4314-abc3-fecd3bc20468-kube-api-access-4l94h\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:08.974244 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:08.974248 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b617f699-e76c-4314-abc3-fecd3bc20468-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:09.670614 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:09.670578 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" event={"ID":"b617f699-e76c-4314-abc3-fecd3bc20468","Type":"ContainerDied","Data":"681a2097d78bff10392619ac9ba57b9103c13125e362a7bb9e427f87f7403dc8"} Jul 16 16:32:09.670614 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:09.670604 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q" Jul 16 16:32:09.671079 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:09.670630 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="681a2097d78bff10392619ac9ba57b9103c13125e362a7bb9e427f87f7403dc8" Jul 16 16:32:32.123743 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:32.123702 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58c8588fd5-m485w"] Jul 16 16:32:57.144921 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.144873 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58c8588fd5-m485w" podUID="ef0f702a-2d09-4729-a88c-878c4087ba8e" containerName="console" containerID="cri-o://ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e" gracePeriod=15 Jul 16 16:32:57.402415 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.402349 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:32:57.495560 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495523 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-config\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.495755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495591 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tlhp\" (UniqueName: \"kubernetes.io/projected/ef0f702a-2d09-4729-a88c-878c4087ba8e-kube-api-access-9tlhp\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.495755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495626 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-service-ca\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.495755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495688 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-oauth-serving-cert\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.495755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495713 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-trusted-ca-bundle\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.495755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495742 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-serving-cert\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.496000 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495769 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-oauth-config\") pod \"ef0f702a-2d09-4729-a88c-878c4087ba8e\" (UID: \"ef0f702a-2d09-4729-a88c-878c4087ba8e\") " Jul 16 16:32:57.496089 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.495953 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-config" (OuterVolumeSpecName: "console-config") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:32:57.496195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.496169 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-service-ca" (OuterVolumeSpecName: "service-ca") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:32:57.496254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.496205 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:32:57.496254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.496234 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:32:57.498041 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.498005 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:32:57.498041 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.498020 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:32:57.498242 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.498071 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef0f702a-2d09-4729-a88c-878c4087ba8e-kube-api-access-9tlhp" (OuterVolumeSpecName: "kube-api-access-9tlhp") pod "ef0f702a-2d09-4729-a88c-878c4087ba8e" (UID: "ef0f702a-2d09-4729-a88c-878c4087ba8e"). InnerVolumeSpecName "kube-api-access-9tlhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:32:57.596839 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596798 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tlhp\" (UniqueName: \"kubernetes.io/projected/ef0f702a-2d09-4729-a88c-878c4087ba8e-kube-api-access-9tlhp\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.596839 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596835 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.596839 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596844 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-oauth-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.597123 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596853 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-trusted-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.597123 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596861 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.597123 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596870 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-oauth-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.597123 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.596879 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef0f702a-2d09-4729-a88c-878c4087ba8e-console-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:32:57.905767 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.905730 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef0f702a-2d09-4729-a88c-878c4087ba8e" containerID="ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e" exitCode=0 Jul 16 16:32:57.905767 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.905772 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c8588fd5-m485w" Jul 16 16:32:57.905994 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.905820 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c8588fd5-m485w" event={"ID":"ef0f702a-2d09-4729-a88c-878c4087ba8e","Type":"ContainerDied","Data":"ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e"} Jul 16 16:32:57.905994 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.905871 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c8588fd5-m485w" event={"ID":"ef0f702a-2d09-4729-a88c-878c4087ba8e","Type":"ContainerDied","Data":"7980d80627d5204d6ed9afce8e33508037fc3305d8b4cc5a558f8a0346fb5d11"} Jul 16 16:32:57.905994 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.905892 2560 scope.go:117] "RemoveContainer" containerID="ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e" Jul 16 16:32:57.915359 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.915342 2560 scope.go:117] "RemoveContainer" containerID="ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e" Jul 16 16:32:57.915611 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:32:57.915592 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e\": container with ID starting with ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e not found: ID does not exist" containerID="ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e" Jul 16 16:32:57.915673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.915624 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e"} err="failed to get container status \"ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e\": rpc error: code = NotFound desc = could not find container \"ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e\": container with ID starting with ab142c0711e26be37eb857c88fa39b5e9a1f9407a311028b254a0cc2b2d1e40e not found: ID does not exist" Jul 16 16:32:57.934245 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.934216 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58c8588fd5-m485w"] Jul 16 16:32:57.940497 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:57.940471 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58c8588fd5-m485w"] Jul 16 16:32:58.656189 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:32:58.656159 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef0f702a-2d09-4729-a88c-878c4087ba8e" path="/var/lib/kubelet/pods/ef0f702a-2d09-4729-a88c-878c4087ba8e/volumes" Jul 16 16:33:34.820076 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820035 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cqr5n"] Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820357 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820369 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820378 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820383 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820391 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="extract" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820396 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="extract" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820404 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef0f702a-2d09-4729-a88c-878c4087ba8e" containerName="console" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820409 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef0f702a-2d09-4729-a88c-878c4087ba8e" containerName="console" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820418 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="pull" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820422 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="pull" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820429 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820434 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820440 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="pull" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820444 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="pull" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820456 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="pull" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820462 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="pull" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820468 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820473 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="util" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820478 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="extract" Jul 16 16:33:34.820475 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820483 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820488 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820492 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820500 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="pull" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820505 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="pull" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820512 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820516 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820562 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9784c48-00dc-4a0e-a9da-43cb4886c638" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820570 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b617f699-e76c-4314-abc3-fecd3bc20468" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820575 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f49632-cded-4ff8-b203-f9b404405bac" containerName="extract" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820580 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef0f702a-2d09-4729-a88c-878c4087ba8e" containerName="console" Jul 16 16:33:34.821043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.820586 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e8fe052-d9c9-4c37-b6b7-f5f2252d6673" containerName="extract" Jul 16 16:33:34.824557 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.824538 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:34.827164 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.827143 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 16:33:34.827537 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.827521 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5s75k\"" Jul 16 16:33:34.827947 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.827929 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 16:33:34.837253 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.837225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cqr5n"] Jul 16 16:33:34.899502 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.899470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xwhz\" (UniqueName: \"kubernetes.io/projected/38591d5d-d3eb-4d05-ac23-fde274dfb273-kube-api-access-2xwhz\") pod \"authorino-58cd955675-cqr5n\" (UID: \"38591d5d-d3eb-4d05-ac23-fde274dfb273\") " pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:34.981375 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.981335 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-g9zwh"] Jul 16 16:33:34.984547 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.984524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:33:34.998600 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:34.998575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g9zwh"] Jul 16 16:33:35.000243 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.000222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xwhz\" (UniqueName: \"kubernetes.io/projected/38591d5d-d3eb-4d05-ac23-fde274dfb273-kube-api-access-2xwhz\") pod \"authorino-58cd955675-cqr5n\" (UID: \"38591d5d-d3eb-4d05-ac23-fde274dfb273\") " pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:35.011638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.011618 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xwhz\" (UniqueName: \"kubernetes.io/projected/38591d5d-d3eb-4d05-ac23-fde274dfb273-kube-api-access-2xwhz\") pod \"authorino-58cd955675-cqr5n\" (UID: \"38591d5d-d3eb-4d05-ac23-fde274dfb273\") " pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:35.101084 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.100970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtm99\" (UniqueName: \"kubernetes.io/projected/7c426c94-1302-45dd-b5e9-34748593ebdb-kube-api-access-gtm99\") pod \"authorino-75d556bd77-g9zwh\" (UID: \"7c426c94-1302-45dd-b5e9-34748593ebdb\") " pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:33:35.133927 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.133894 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:35.201777 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.201742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtm99\" (UniqueName: \"kubernetes.io/projected/7c426c94-1302-45dd-b5e9-34748593ebdb-kube-api-access-gtm99\") pod \"authorino-75d556bd77-g9zwh\" (UID: \"7c426c94-1302-45dd-b5e9-34748593ebdb\") " pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:33:35.211892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.211859 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtm99\" (UniqueName: \"kubernetes.io/projected/7c426c94-1302-45dd-b5e9-34748593ebdb-kube-api-access-gtm99\") pod \"authorino-75d556bd77-g9zwh\" (UID: \"7c426c94-1302-45dd-b5e9-34748593ebdb\") " pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:33:35.284090 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.283600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cqr5n"] Jul 16 16:33:35.287920 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:33:35.287871 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38591d5d_d3eb_4d05_ac23_fde274dfb273.slice/crio-46897584d5edfcff4d9a7b525cafadc2690b5943a4bdf68f6fe445b9f2d16b51 WatchSource:0}: Error finding container 46897584d5edfcff4d9a7b525cafadc2690b5943a4bdf68f6fe445b9f2d16b51: Status 404 returned error can't find the container with id 46897584d5edfcff4d9a7b525cafadc2690b5943a4bdf68f6fe445b9f2d16b51 Jul 16 16:33:35.293668 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.293642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:33:35.431259 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:35.431234 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g9zwh"] Jul 16 16:33:35.432576 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:33:35.432546 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c426c94_1302_45dd_b5e9_34748593ebdb.slice/crio-ffb5eb6d98b04ae75740382d0f95ea4f12efb6bd0aa8c6a662c926220217dcbd WatchSource:0}: Error finding container ffb5eb6d98b04ae75740382d0f95ea4f12efb6bd0aa8c6a662c926220217dcbd: Status 404 returned error can't find the container with id ffb5eb6d98b04ae75740382d0f95ea4f12efb6bd0aa8c6a662c926220217dcbd Jul 16 16:33:36.101268 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:36.101226 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g9zwh" event={"ID":"7c426c94-1302-45dd-b5e9-34748593ebdb","Type":"ContainerStarted","Data":"ffb5eb6d98b04ae75740382d0f95ea4f12efb6bd0aa8c6a662c926220217dcbd"} Jul 16 16:33:36.102724 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:36.102691 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cqr5n" event={"ID":"38591d5d-d3eb-4d05-ac23-fde274dfb273","Type":"ContainerStarted","Data":"46897584d5edfcff4d9a7b525cafadc2690b5943a4bdf68f6fe445b9f2d16b51"} Jul 16 16:33:40.132882 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:40.132843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g9zwh" event={"ID":"7c426c94-1302-45dd-b5e9-34748593ebdb","Type":"ContainerStarted","Data":"ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25"} Jul 16 16:33:40.134444 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:40.134420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cqr5n" event={"ID":"38591d5d-d3eb-4d05-ac23-fde274dfb273","Type":"ContainerStarted","Data":"9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574"} Jul 16 16:33:40.151724 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:40.151673 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-g9zwh" podStartSLOduration=2.5176258430000003 podStartE2EDuration="6.151656446s" totalImagesPullingTime="3.634030603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:33:40.150862942 +0000 UTC m=+495.958974957" watchObservedRunningTime="2026-07-16 16:33:40.151656446 +0000 UTC m=+495.959768451" Jul 16 16:33:40.168946 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:40.168895 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cqr5n" podStartSLOduration=2.401767254 podStartE2EDuration="6.168878906s" totalImagesPullingTime="3.767111652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:33:40.166770963 +0000 UTC m=+495.974882978" watchObservedRunningTime="2026-07-16 16:33:40.168878906 +0000 UTC m=+495.976990921" Jul 16 16:33:40.197265 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:40.197229 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cqr5n"] Jul 16 16:33:42.144029 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:42.143987 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cqr5n" podUID="38591d5d-d3eb-4d05-ac23-fde274dfb273" containerName="authorino" containerID="cri-o://9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574" gracePeriod=30 Jul 16 16:33:42.402218 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:42.402157 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:42.468569 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:42.468530 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xwhz\" (UniqueName: \"kubernetes.io/projected/38591d5d-d3eb-4d05-ac23-fde274dfb273-kube-api-access-2xwhz\") pod \"38591d5d-d3eb-4d05-ac23-fde274dfb273\" (UID: \"38591d5d-d3eb-4d05-ac23-fde274dfb273\") " Jul 16 16:33:42.470726 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:42.470697 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38591d5d-d3eb-4d05-ac23-fde274dfb273-kube-api-access-2xwhz" (OuterVolumeSpecName: "kube-api-access-2xwhz") pod "38591d5d-d3eb-4d05-ac23-fde274dfb273" (UID: "38591d5d-d3eb-4d05-ac23-fde274dfb273"). InnerVolumeSpecName "kube-api-access-2xwhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:33:42.569491 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:42.569457 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xwhz\" (UniqueName: \"kubernetes.io/projected/38591d5d-d3eb-4d05-ac23-fde274dfb273-kube-api-access-2xwhz\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:33:43.150556 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.150517 2560 generic.go:356] "Generic (PLEG): container finished" podID="38591d5d-d3eb-4d05-ac23-fde274dfb273" containerID="9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574" exitCode=0 Jul 16 16:33:43.150958 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.150561 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cqr5n" Jul 16 16:33:43.150958 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.150601 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cqr5n" event={"ID":"38591d5d-d3eb-4d05-ac23-fde274dfb273","Type":"ContainerDied","Data":"9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574"} Jul 16 16:33:43.150958 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.150640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cqr5n" event={"ID":"38591d5d-d3eb-4d05-ac23-fde274dfb273","Type":"ContainerDied","Data":"46897584d5edfcff4d9a7b525cafadc2690b5943a4bdf68f6fe445b9f2d16b51"} Jul 16 16:33:43.150958 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.150660 2560 scope.go:117] "RemoveContainer" containerID="9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574" Jul 16 16:33:43.159688 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.159665 2560 scope.go:117] "RemoveContainer" containerID="9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574" Jul 16 16:33:43.159988 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:33:43.159966 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574\": container with ID starting with 9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574 not found: ID does not exist" containerID="9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574" Jul 16 16:33:43.160093 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.160003 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574"} err="failed to get container status \"9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574\": rpc error: code = NotFound desc = could not find container \"9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574\": container with ID starting with 9ef61e0a3a90ecca57231972906df10e5eb0cb96d68a2e857678b8ee60130574 not found: ID does not exist" Jul 16 16:33:43.170727 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.170702 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cqr5n"] Jul 16 16:33:43.178908 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:43.178885 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cqr5n"] Jul 16 16:33:44.655026 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:33:44.654992 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38591d5d-d3eb-4d05-ac23-fde274dfb273" path="/var/lib/kubelet/pods/38591d5d-d3eb-4d05-ac23-fde274dfb273/volumes" Jul 16 16:34:03.617416 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.617377 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kjjmn"] Jul 16 16:34:03.617783 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.617729 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38591d5d-d3eb-4d05-ac23-fde274dfb273" containerName="authorino" Jul 16 16:34:03.617783 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.617740 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38591d5d-d3eb-4d05-ac23-fde274dfb273" containerName="authorino" Jul 16 16:34:03.617852 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.617806 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38591d5d-d3eb-4d05-ac23-fde274dfb273" containerName="authorino" Jul 16 16:34:03.620803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.620787 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:03.630755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.630728 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kjjmn"] Jul 16 16:34:03.751595 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.751548 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8v54\" (UniqueName: \"kubernetes.io/projected/287f5d44-fa18-42da-acfb-97d5a2d6926c-kube-api-access-f8v54\") pod \"authorino-5d4b67c446-kjjmn\" (UID: \"287f5d44-fa18-42da-acfb-97d5a2d6926c\") " pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:03.852360 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.852311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8v54\" (UniqueName: \"kubernetes.io/projected/287f5d44-fa18-42da-acfb-97d5a2d6926c-kube-api-access-f8v54\") pod \"authorino-5d4b67c446-kjjmn\" (UID: \"287f5d44-fa18-42da-acfb-97d5a2d6926c\") " pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:03.860573 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.860549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8v54\" (UniqueName: \"kubernetes.io/projected/287f5d44-fa18-42da-acfb-97d5a2d6926c-kube-api-access-f8v54\") pod \"authorino-5d4b67c446-kjjmn\" (UID: \"287f5d44-fa18-42da-acfb-97d5a2d6926c\") " pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:03.865019 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.864992 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kjjmn"] Jul 16 16:34:03.865249 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.865237 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:03.901661 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.901629 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-695c54d594-749hl"] Jul 16 16:34:03.905020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.905001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:03.916013 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:03.915943 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-695c54d594-749hl"] Jul 16 16:34:04.006644 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.006619 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kjjmn"] Jul 16 16:34:04.009079 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:34:04.009024 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod287f5d44_fa18_42da_acfb_97d5a2d6926c.slice/crio-8c7f008c23aecdb9799bc5aafe04895d696bf2346ff0de08e6f48d298b7153a3 WatchSource:0}: Error finding container 8c7f008c23aecdb9799bc5aafe04895d696bf2346ff0de08e6f48d298b7153a3: Status 404 returned error can't find the container with id 8c7f008c23aecdb9799bc5aafe04895d696bf2346ff0de08e6f48d298b7153a3 Jul 16 16:34:04.054172 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.054131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzqht\" (UniqueName: \"kubernetes.io/projected/68473aaf-2b2b-4286-a4aa-951c245eecbc-kube-api-access-kzqht\") pod \"authorino-695c54d594-749hl\" (UID: \"68473aaf-2b2b-4286-a4aa-951c245eecbc\") " pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:04.155329 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.155225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzqht\" (UniqueName: \"kubernetes.io/projected/68473aaf-2b2b-4286-a4aa-951c245eecbc-kube-api-access-kzqht\") pod \"authorino-695c54d594-749hl\" (UID: \"68473aaf-2b2b-4286-a4aa-951c245eecbc\") " pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:04.165135 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.165105 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzqht\" (UniqueName: \"kubernetes.io/projected/68473aaf-2b2b-4286-a4aa-951c245eecbc-kube-api-access-kzqht\") pod \"authorino-695c54d594-749hl\" (UID: \"68473aaf-2b2b-4286-a4aa-951c245eecbc\") " pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:04.216091 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.216030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:04.223755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.223724 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-695c54d594-749hl"] Jul 16 16:34:04.254575 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.254543 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c675c59db-64c9m"] Jul 16 16:34:04.259104 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.258112 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.261482 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.261453 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 16 16:34:04.263754 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.263726 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" event={"ID":"287f5d44-fa18-42da-acfb-97d5a2d6926c","Type":"ContainerStarted","Data":"8c7f008c23aecdb9799bc5aafe04895d696bf2346ff0de08e6f48d298b7153a3"} Jul 16 16:34:04.267638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.267616 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c675c59db-64c9m"] Jul 16 16:34:04.357417 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.357351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e206b7da-f819-4461-8255-559045af3a24-tls-cert\") pod \"authorino-c675c59db-64c9m\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.357417 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.357397 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjstn\" (UniqueName: \"kubernetes.io/projected/e206b7da-f819-4461-8255-559045af3a24-kube-api-access-gjstn\") pod \"authorino-c675c59db-64c9m\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.359809 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.359784 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-695c54d594-749hl"] Jul 16 16:34:04.362828 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:34:04.362804 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68473aaf_2b2b_4286_a4aa_951c245eecbc.slice/crio-8840a9031d430df7d6001b1f3d2ff1825032548c5508d24a94d451a60f1b28aa WatchSource:0}: Error finding container 8840a9031d430df7d6001b1f3d2ff1825032548c5508d24a94d451a60f1b28aa: Status 404 returned error can't find the container with id 8840a9031d430df7d6001b1f3d2ff1825032548c5508d24a94d451a60f1b28aa Jul 16 16:34:04.457970 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.457927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e206b7da-f819-4461-8255-559045af3a24-tls-cert\") pod \"authorino-c675c59db-64c9m\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.458149 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.457985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjstn\" (UniqueName: \"kubernetes.io/projected/e206b7da-f819-4461-8255-559045af3a24-kube-api-access-gjstn\") pod \"authorino-c675c59db-64c9m\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.460610 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.460581 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e206b7da-f819-4461-8255-559045af3a24-tls-cert\") pod \"authorino-c675c59db-64c9m\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.466130 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.466106 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjstn\" (UniqueName: \"kubernetes.io/projected/e206b7da-f819-4461-8255-559045af3a24-kube-api-access-gjstn\") pod \"authorino-c675c59db-64c9m\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.570677 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.570630 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:34:04.708229 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:04.708203 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c675c59db-64c9m"] Jul 16 16:34:04.710769 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:34:04.710741 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode206b7da_f819_4461_8255_559045af3a24.slice/crio-35aa956f4a17bc277c8239101fd98eb4be79d49090fa3b24503a3856378de25d WatchSource:0}: Error finding container 35aa956f4a17bc277c8239101fd98eb4be79d49090fa3b24503a3856378de25d: Status 404 returned error can't find the container with id 35aa956f4a17bc277c8239101fd98eb4be79d49090fa3b24503a3856378de25d Jul 16 16:34:05.270888 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.270856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" event={"ID":"287f5d44-fa18-42da-acfb-97d5a2d6926c","Type":"ContainerStarted","Data":"dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f"} Jul 16 16:34:05.271109 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.270936 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" podUID="287f5d44-fa18-42da-acfb-97d5a2d6926c" containerName="authorino" containerID="cri-o://dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f" gracePeriod=30 Jul 16 16:34:05.272517 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.272489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c675c59db-64c9m" event={"ID":"e206b7da-f819-4461-8255-559045af3a24","Type":"ContainerStarted","Data":"a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8"} Jul 16 16:34:05.272517 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.272518 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c675c59db-64c9m" event={"ID":"e206b7da-f819-4461-8255-559045af3a24","Type":"ContainerStarted","Data":"35aa956f4a17bc277c8239101fd98eb4be79d49090fa3b24503a3856378de25d"} Jul 16 16:34:05.274032 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.274011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695c54d594-749hl" event={"ID":"68473aaf-2b2b-4286-a4aa-951c245eecbc","Type":"ContainerStarted","Data":"e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353"} Jul 16 16:34:05.274156 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.274041 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695c54d594-749hl" event={"ID":"68473aaf-2b2b-4286-a4aa-951c245eecbc","Type":"ContainerStarted","Data":"8840a9031d430df7d6001b1f3d2ff1825032548c5508d24a94d451a60f1b28aa"} Jul 16 16:34:05.274156 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.274090 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-695c54d594-749hl" podUID="68473aaf-2b2b-4286-a4aa-951c245eecbc" containerName="authorino" containerID="cri-o://e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353" gracePeriod=30 Jul 16 16:34:05.297804 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.297755 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" podStartSLOduration=1.9433212119999999 podStartE2EDuration="2.297739663s" totalImagesPullingTime="354.418451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:34:05.296680644 +0000 UTC m=+521.104792658" watchObservedRunningTime="2026-07-16 16:34:05.297739663 +0000 UTC m=+521.105851678" Jul 16 16:34:05.320902 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.320801 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-695c54d594-749hl" podStartSLOduration=1.857045298 podStartE2EDuration="2.320783832s" totalImagesPullingTime="463.738534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:34:05.31750652 +0000 UTC m=+521.125618535" watchObservedRunningTime="2026-07-16 16:34:05.320783832 +0000 UTC m=+521.128895846" Jul 16 16:34:05.372925 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.372867 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c675c59db-64c9m" podStartSLOduration=1.026089656 podStartE2EDuration="1.372844742s" totalImagesPullingTime="346.755086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:34:05.341520534 +0000 UTC m=+521.149632549" watchObservedRunningTime="2026-07-16 16:34:05.372844742 +0000 UTC m=+521.180956756" Jul 16 16:34:05.374640 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.374613 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g9zwh"] Jul 16 16:34:05.374876 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.374845 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-g9zwh" podUID="7c426c94-1302-45dd-b5e9-34748593ebdb" containerName="authorino" containerID="cri-o://ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25" gracePeriod=30 Jul 16 16:34:05.564010 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.563773 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:05.574493 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.574467 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:05.638916 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.638889 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:34:05.667940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.667902 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8v54\" (UniqueName: \"kubernetes.io/projected/287f5d44-fa18-42da-acfb-97d5a2d6926c-kube-api-access-f8v54\") pod \"287f5d44-fa18-42da-acfb-97d5a2d6926c\" (UID: \"287f5d44-fa18-42da-acfb-97d5a2d6926c\") " Jul 16 16:34:05.668143 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.668029 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzqht\" (UniqueName: \"kubernetes.io/projected/68473aaf-2b2b-4286-a4aa-951c245eecbc-kube-api-access-kzqht\") pod \"68473aaf-2b2b-4286-a4aa-951c245eecbc\" (UID: \"68473aaf-2b2b-4286-a4aa-951c245eecbc\") " Jul 16 16:34:05.670182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.670147 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68473aaf-2b2b-4286-a4aa-951c245eecbc-kube-api-access-kzqht" (OuterVolumeSpecName: "kube-api-access-kzqht") pod "68473aaf-2b2b-4286-a4aa-951c245eecbc" (UID: "68473aaf-2b2b-4286-a4aa-951c245eecbc"). InnerVolumeSpecName "kube-api-access-kzqht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:34:05.670302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.670152 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287f5d44-fa18-42da-acfb-97d5a2d6926c-kube-api-access-f8v54" (OuterVolumeSpecName: "kube-api-access-f8v54") pod "287f5d44-fa18-42da-acfb-97d5a2d6926c" (UID: "287f5d44-fa18-42da-acfb-97d5a2d6926c"). InnerVolumeSpecName "kube-api-access-f8v54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:34:05.769325 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.769290 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtm99\" (UniqueName: \"kubernetes.io/projected/7c426c94-1302-45dd-b5e9-34748593ebdb-kube-api-access-gtm99\") pod \"7c426c94-1302-45dd-b5e9-34748593ebdb\" (UID: \"7c426c94-1302-45dd-b5e9-34748593ebdb\") " Jul 16 16:34:05.769756 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.769529 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzqht\" (UniqueName: \"kubernetes.io/projected/68473aaf-2b2b-4286-a4aa-951c245eecbc-kube-api-access-kzqht\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:34:05.769756 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.769546 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8v54\" (UniqueName: \"kubernetes.io/projected/287f5d44-fa18-42da-acfb-97d5a2d6926c-kube-api-access-f8v54\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:34:05.771781 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.771758 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c426c94-1302-45dd-b5e9-34748593ebdb-kube-api-access-gtm99" (OuterVolumeSpecName: "kube-api-access-gtm99") pod "7c426c94-1302-45dd-b5e9-34748593ebdb" (UID: "7c426c94-1302-45dd-b5e9-34748593ebdb"). InnerVolumeSpecName "kube-api-access-gtm99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:34:05.870092 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:05.870048 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtm99\" (UniqueName: \"kubernetes.io/projected/7c426c94-1302-45dd-b5e9-34748593ebdb-kube-api-access-gtm99\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:34:06.279912 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.279876 2560 generic.go:356] "Generic (PLEG): container finished" podID="7c426c94-1302-45dd-b5e9-34748593ebdb" containerID="ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25" exitCode=0 Jul 16 16:34:06.280137 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.279920 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g9zwh" Jul 16 16:34:06.280137 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.279962 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g9zwh" event={"ID":"7c426c94-1302-45dd-b5e9-34748593ebdb","Type":"ContainerDied","Data":"ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25"} Jul 16 16:34:06.280137 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.280001 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g9zwh" event={"ID":"7c426c94-1302-45dd-b5e9-34748593ebdb","Type":"ContainerDied","Data":"ffb5eb6d98b04ae75740382d0f95ea4f12efb6bd0aa8c6a662c926220217dcbd"} Jul 16 16:34:06.280137 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.280023 2560 scope.go:117] "RemoveContainer" containerID="ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25" Jul 16 16:34:06.281362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.281239 2560 generic.go:356] "Generic (PLEG): container finished" podID="287f5d44-fa18-42da-acfb-97d5a2d6926c" containerID="dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f" exitCode=0 Jul 16 16:34:06.281362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.281273 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" Jul 16 16:34:06.281362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.281312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" event={"ID":"287f5d44-fa18-42da-acfb-97d5a2d6926c","Type":"ContainerDied","Data":"dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f"} Jul 16 16:34:06.281362 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.281338 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kjjmn" event={"ID":"287f5d44-fa18-42da-acfb-97d5a2d6926c","Type":"ContainerDied","Data":"8c7f008c23aecdb9799bc5aafe04895d696bf2346ff0de08e6f48d298b7153a3"} Jul 16 16:34:06.282826 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.282805 2560 generic.go:356] "Generic (PLEG): container finished" podID="68473aaf-2b2b-4286-a4aa-951c245eecbc" containerID="e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353" exitCode=0 Jul 16 16:34:06.282954 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.282843 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695c54d594-749hl" Jul 16 16:34:06.282954 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.282869 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695c54d594-749hl" event={"ID":"68473aaf-2b2b-4286-a4aa-951c245eecbc","Type":"ContainerDied","Data":"e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353"} Jul 16 16:34:06.282954 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.282894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695c54d594-749hl" event={"ID":"68473aaf-2b2b-4286-a4aa-951c245eecbc","Type":"ContainerDied","Data":"8840a9031d430df7d6001b1f3d2ff1825032548c5508d24a94d451a60f1b28aa"} Jul 16 16:34:06.289769 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.289744 2560 scope.go:117] "RemoveContainer" containerID="ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25" Jul 16 16:34:06.290090 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:06.290040 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25\": container with ID starting with ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25 not found: ID does not exist" containerID="ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25" Jul 16 16:34:06.290206 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.290094 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25"} err="failed to get container status \"ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25\": rpc error: code = NotFound desc = could not find container \"ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25\": container with ID starting with ee3584ddc6f392431261c813041b11e73d31ff760df3148371ac414e2820cc25 not found: ID does not exist" Jul 16 16:34:06.290206 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.290116 2560 scope.go:117] "RemoveContainer" containerID="dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f" Jul 16 16:34:06.298883 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.298866 2560 scope.go:117] "RemoveContainer" containerID="dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f" Jul 16 16:34:06.299177 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:06.299156 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f\": container with ID starting with dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f not found: ID does not exist" containerID="dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f" Jul 16 16:34:06.299232 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.299186 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f"} err="failed to get container status \"dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f\": rpc error: code = NotFound desc = could not find container \"dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f\": container with ID starting with dc94a17f318f3278b4b898ab6ea5bf88ead7b48b739ce1e0bc9afc0b3c62b26f not found: ID does not exist" Jul 16 16:34:06.299232 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.299202 2560 scope.go:117] "RemoveContainer" containerID="e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353" Jul 16 16:34:06.307336 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.307313 2560 scope.go:117] "RemoveContainer" containerID="e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353" Jul 16 16:34:06.307610 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:06.307591 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353\": container with ID starting with e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353 not found: ID does not exist" containerID="e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353" Jul 16 16:34:06.307668 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.307617 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353"} err="failed to get container status \"e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353\": rpc error: code = NotFound desc = could not find container \"e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353\": container with ID starting with e8554c4c3aef2e7b82caf7d1410c59f60e05a756a8f60ac45722726ffd214353 not found: ID does not exist" Jul 16 16:34:06.323808 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.323779 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kjjmn"] Jul 16 16:34:06.325597 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.325575 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kjjmn"] Jul 16 16:34:06.336755 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.336727 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-695c54d594-749hl"] Jul 16 16:34:06.346751 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.346726 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-695c54d594-749hl"] Jul 16 16:34:06.363468 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.363441 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g9zwh"] Jul 16 16:34:06.366085 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.366047 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g9zwh"] Jul 16 16:34:06.655896 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.655815 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287f5d44-fa18-42da-acfb-97d5a2d6926c" path="/var/lib/kubelet/pods/287f5d44-fa18-42da-acfb-97d5a2d6926c/volumes" Jul 16 16:34:06.656196 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.656181 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68473aaf-2b2b-4286-a4aa-951c245eecbc" path="/var/lib/kubelet/pods/68473aaf-2b2b-4286-a4aa-951c245eecbc/volumes" Jul 16 16:34:06.656488 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:06.656476 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c426c94-1302-45dd-b5e9-34748593ebdb" path="/var/lib/kubelet/pods/7c426c94-1302-45dd-b5e9-34748593ebdb/volumes" Jul 16 16:34:10.629307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629277 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67c787bf8d-ktcm9"] Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629621 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287f5d44-fa18-42da-acfb-97d5a2d6926c" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629635 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="287f5d44-fa18-42da-acfb-97d5a2d6926c" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629658 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68473aaf-2b2b-4286-a4aa-951c245eecbc" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629663 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="68473aaf-2b2b-4286-a4aa-951c245eecbc" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629675 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c426c94-1302-45dd-b5e9-34748593ebdb" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629681 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c426c94-1302-45dd-b5e9-34748593ebdb" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629728 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c426c94-1302-45dd-b5e9-34748593ebdb" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629736 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="68473aaf-2b2b-4286-a4aa-951c245eecbc" containerName="authorino" Jul 16 16:34:10.629813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.629743 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="287f5d44-fa18-42da-acfb-97d5a2d6926c" containerName="authorino" Jul 16 16:34:10.634560 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.634539 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:10.636917 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.636890 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 16:34:10.637293 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.637275 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dcbvg\"" Jul 16 16:34:10.637524 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.637508 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 16:34:10.644494 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.644469 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c787bf8d-ktcm9"] Jul 16 16:34:10.812614 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.812581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98ppn\" (UniqueName: \"kubernetes.io/projected/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-kube-api-access-98ppn\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:10.812809 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.812632 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:10.824034 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.824001 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6847bf9b68-mj7vf"] Jul 16 16:34:10.826796 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.826766 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:10.841625 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.841599 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-mj7vf"] Jul 16 16:34:10.914190 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.914097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxwnm\" (UniqueName: \"kubernetes.io/projected/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-kube-api-access-vxwnm\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:10.914371 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.914198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98ppn\" (UniqueName: \"kubernetes.io/projected/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-kube-api-access-98ppn\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:10.914371 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.914244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:10.914371 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.914275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:10.914527 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:10.914375 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 16 16:34:10.914527 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:10.914427 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert podName:c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2 nodeName:}" failed. No retries permitted until 2026-07-16 16:34:11.414411885 +0000 UTC m=+527.222523877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert") pod "maas-controller-67c787bf8d-ktcm9" (UID: "c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2") : secret "maas-controller-webhook-cert" not found Jul 16 16:34:10.922815 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:10.922781 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98ppn\" (UniqueName: \"kubernetes.io/projected/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-kube-api-access-98ppn\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:11.015396 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.015349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:11.015603 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.015423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxwnm\" (UniqueName: \"kubernetes.io/projected/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-kube-api-access-vxwnm\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:11.015603 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:11.015501 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 16 16:34:11.015603 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:11.015564 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert podName:198df1fa-8ccb-45a4-ad24-43ea5a9f13c2 nodeName:}" failed. No retries permitted until 2026-07-16 16:34:11.515549531 +0000 UTC m=+527.323661524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert") pod "maas-controller-6847bf9b68-mj7vf" (UID: "198df1fa-8ccb-45a4-ad24-43ea5a9f13c2") : secret "maas-controller-webhook-cert" not found Jul 16 16:34:11.024613 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.024582 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxwnm\" (UniqueName: \"kubernetes.io/projected/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-kube-api-access-vxwnm\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:11.417721 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.417681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:11.420273 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.420253 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert\") pod \"maas-controller-67c787bf8d-ktcm9\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:11.518673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.518633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:11.521219 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.521200 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert\") pod \"maas-controller-6847bf9b68-mj7vf\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:11.546268 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.546233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:11.686176 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.686151 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c787bf8d-ktcm9"] Jul 16 16:34:11.688146 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:34:11.688097 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93e05c1_7fa8_42a1_9dbd_6198c2b3d8c2.slice/crio-7960d741e571053c64953d4ba887d78986e3b4541c45f1cb7a9cade3df90ea70 WatchSource:0}: Error finding container 7960d741e571053c64953d4ba887d78986e3b4541c45f1cb7a9cade3df90ea70: Status 404 returned error can't find the container with id 7960d741e571053c64953d4ba887d78986e3b4541c45f1cb7a9cade3df90ea70 Jul 16 16:34:11.736937 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.736897 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:11.875338 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:11.875303 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-mj7vf"] Jul 16 16:34:11.878263 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:34:11.878238 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198df1fa_8ccb_45a4_ad24_43ea5a9f13c2.slice/crio-321a1cf312f591a8e9ae0808fa5bf07a5851c866d37556d5e614b1ba0b5108fe WatchSource:0}: Error finding container 321a1cf312f591a8e9ae0808fa5bf07a5851c866d37556d5e614b1ba0b5108fe: Status 404 returned error can't find the container with id 321a1cf312f591a8e9ae0808fa5bf07a5851c866d37556d5e614b1ba0b5108fe Jul 16 16:34:12.326668 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:12.326617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" event={"ID":"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2","Type":"ContainerStarted","Data":"321a1cf312f591a8e9ae0808fa5bf07a5851c866d37556d5e614b1ba0b5108fe"} Jul 16 16:34:12.328316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:12.328278 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" event={"ID":"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2","Type":"ContainerStarted","Data":"7960d741e571053c64953d4ba887d78986e3b4541c45f1cb7a9cade3df90ea70"} Jul 16 16:34:15.351525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:15.351300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" event={"ID":"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2","Type":"ContainerStarted","Data":"3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3"} Jul 16 16:34:15.351525 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:15.351390 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:15.373929 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:15.373868 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" podStartSLOduration=2.7876648619999997 podStartE2EDuration="5.373848435s" totalImagesPullingTime="2.586183573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:34:15.371528671 +0000 UTC m=+531.179640686" watchObservedRunningTime="2026-07-16 16:34:15.373848435 +0000 UTC m=+531.181960452" Jul 16 16:34:16.358263 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:16.358223 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" event={"ID":"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2","Type":"ContainerStarted","Data":"0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d"} Jul 16 16:34:16.358753 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:16.358550 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:16.378616 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:16.378554 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" podStartSLOduration=2.82766426 podStartE2EDuration="6.378536155s" totalImagesPullingTime="3.550871895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:34:16.376816625 +0000 UTC m=+532.184928653" watchObservedRunningTime="2026-07-16 16:34:16.378536155 +0000 UTC m=+532.186648236" Jul 16 16:34:26.361070 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:26.361023 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:28.371074 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.371037 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:34:28.425051 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.425021 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67c787bf8d-ktcm9"] Jul 16 16:34:28.425258 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.425235 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" podUID="c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" containerName="manager" containerID="cri-o://3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3" gracePeriod=30 Jul 16 16:34:28.671654 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.671629 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:28.754337 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.754301 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert\") pod \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " Jul 16 16:34:28.754527 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.754351 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98ppn\" (UniqueName: \"kubernetes.io/projected/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-kube-api-access-98ppn\") pod \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\" (UID: \"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2\") " Jul 16 16:34:28.756739 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.756709 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" (UID: "c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:34:28.756836 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.756747 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-kube-api-access-98ppn" (OuterVolumeSpecName: "kube-api-access-98ppn") pod "c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" (UID: "c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2"). InnerVolumeSpecName "kube-api-access-98ppn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:34:28.854916 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.854883 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-webhook-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:34:28.854916 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:28.854909 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98ppn\" (UniqueName: \"kubernetes.io/projected/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2-kube-api-access-98ppn\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:34:29.426419 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.426380 2560 generic.go:356] "Generic (PLEG): container finished" podID="c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" containerID="3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3" exitCode=0 Jul 16 16:34:29.426852 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.426424 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" Jul 16 16:34:29.426852 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.426442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" event={"ID":"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2","Type":"ContainerDied","Data":"3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3"} Jul 16 16:34:29.426852 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.426492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c787bf8d-ktcm9" event={"ID":"c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2","Type":"ContainerDied","Data":"7960d741e571053c64953d4ba887d78986e3b4541c45f1cb7a9cade3df90ea70"} Jul 16 16:34:29.426852 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.426516 2560 scope.go:117] "RemoveContainer" containerID="3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3" Jul 16 16:34:29.435770 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.435752 2560 scope.go:117] "RemoveContainer" containerID="3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3" Jul 16 16:34:29.436100 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:34:29.436035 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3\": container with ID starting with 3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3 not found: ID does not exist" containerID="3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3" Jul 16 16:34:29.436100 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.436087 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3"} err="failed to get container status \"3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3\": rpc error: code = NotFound desc = could not find container \"3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3\": container with ID starting with 3bd21470bfd5ac90792f0011b9fd03298f8b294d7efe811110a6845cdc482fa3 not found: ID does not exist" Jul 16 16:34:29.446918 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.446894 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67c787bf8d-ktcm9"] Jul 16 16:34:29.453492 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:29.453469 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67c787bf8d-ktcm9"] Jul 16 16:34:30.654658 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:30.654623 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" path="/var/lib/kubelet/pods/c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2/volumes" Jul 16 16:34:47.991364 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.991328 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m"] Jul 16 16:34:47.991861 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.991648 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" containerName="manager" Jul 16 16:34:47.991861 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.991659 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" containerName="manager" Jul 16 16:34:47.991861 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.991711 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93e05c1-7fa8-42a1-9dbd-6198c2b3d8c2" containerName="manager" Jul 16 16:34:47.995734 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.995712 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:47.998215 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.998190 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 16:34:47.998316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.998190 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 16:34:47.998486 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:47.998472 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 16:34:48.011137 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.011112 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m"] Jul 16 16:34:48.112813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.112772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd2a36f-ecf9-42c9-a272-d209a6a27042-maas-api-tls\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.112813 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.112813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd2a36f-ecf9-42c9-a272-d209a6a27042-openshift-service-ca\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.113020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.112832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7ch8\" (UniqueName: \"kubernetes.io/projected/1fd2a36f-ecf9-42c9-a272-d209a6a27042-kube-api-access-b7ch8\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.213899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.213862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd2a36f-ecf9-42c9-a272-d209a6a27042-maas-api-tls\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.213899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.213902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd2a36f-ecf9-42c9-a272-d209a6a27042-openshift-service-ca\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.214130 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.213919 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7ch8\" (UniqueName: \"kubernetes.io/projected/1fd2a36f-ecf9-42c9-a272-d209a6a27042-kube-api-access-b7ch8\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.214543 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.214509 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd2a36f-ecf9-42c9-a272-d209a6a27042-openshift-service-ca\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.216424 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.216398 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd2a36f-ecf9-42c9-a272-d209a6a27042-maas-api-tls\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.223803 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.223780 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7ch8\" (UniqueName: \"kubernetes.io/projected/1fd2a36f-ecf9-42c9-a272-d209a6a27042-kube-api-access-b7ch8\") pod \"maas-api-7664fcc798-bfw2m\" (UID: \"1fd2a36f-ecf9-42c9-a272-d209a6a27042\") " pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.305861 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.305777 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:48.449335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.449297 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m"] Jul 16 16:34:48.452804 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:34:48.452782 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd2a36f_ecf9_42c9_a272_d209a6a27042.slice/crio-4c5b488a3517a0baa056bc746a2d8c7df8f3f2bc2febde14662d09b6314eea3f WatchSource:0}: Error finding container 4c5b488a3517a0baa056bc746a2d8c7df8f3f2bc2febde14662d09b6314eea3f: Status 404 returned error can't find the container with id 4c5b488a3517a0baa056bc746a2d8c7df8f3f2bc2febde14662d09b6314eea3f Jul 16 16:34:48.523123 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:48.523086 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" event={"ID":"1fd2a36f-ecf9-42c9-a272-d209a6a27042","Type":"ContainerStarted","Data":"4c5b488a3517a0baa056bc746a2d8c7df8f3f2bc2febde14662d09b6314eea3f"} Jul 16 16:34:52.544673 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:52.544635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" event={"ID":"1fd2a36f-ecf9-42c9-a272-d209a6a27042","Type":"ContainerStarted","Data":"4755f8d727d0b5aa2b80a1b2ffbe7cbda1f9bcf8a517cee17d8d784076b4d16a"} Jul 16 16:34:52.545157 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:52.544727 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:34:52.564139 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:52.564007 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" podStartSLOduration=1.707665874 podStartE2EDuration="5.563990811s" totalImagesPullingTime="3.856324937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:34:52.562624424 +0000 UTC m=+568.370736438" watchObservedRunningTime="2026-07-16 16:34:52.563990811 +0000 UTC m=+568.372102824" Jul 16 16:34:58.556308 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:34:58.556273 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7664fcc798-bfw2m" Jul 16 16:35:17.933763 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.933728 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp"] Jul 16 16:35:17.936899 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.936874 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:17.939757 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.939730 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p8r4s\"" Jul 16 16:35:17.940710 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.940693 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 16:35:17.940839 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.940819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 16:35:17.940966 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.940947 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 16:35:17.946814 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.946786 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp"] Jul 16 16:35:17.984541 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.984502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9zx5\" (UniqueName: \"kubernetes.io/projected/b36e52a4-3ae2-4b5f-b688-867bae24fb61-kube-api-access-f9zx5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:17.984541 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.984548 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:17.984749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.984597 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:17.984749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.984613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b36e52a4-3ae2-4b5f-b688-867bae24fb61-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:17.984749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.984641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:17.984749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:17.984723 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086018 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.085974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9zx5\" (UniqueName: \"kubernetes.io/projected/b36e52a4-3ae2-4b5f-b688-867bae24fb61-kube-api-access-f9zx5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b36e52a4-3ae2-4b5f-b688-867bae24fb61-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086502 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086560 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086533 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.086634 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.086611 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.088445 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.088425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b36e52a4-3ae2-4b5f-b688-867bae24fb61-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.088756 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.088740 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b36e52a4-3ae2-4b5f-b688-867bae24fb61-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.095527 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.095502 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9zx5\" (UniqueName: \"kubernetes.io/projected/b36e52a4-3ae2-4b5f-b688-867bae24fb61-kube-api-access-f9zx5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xlgsp\" (UID: \"b36e52a4-3ae2-4b5f-b688-867bae24fb61\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.249292 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.249201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:18.382597 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.382576 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp"] Jul 16 16:35:18.384666 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:35:18.384638 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36e52a4_3ae2_4b5f_b688_867bae24fb61.slice/crio-14c4c66fd605ed3fa4938573d9bb5de7481dcb3d9ac3050caea4a9ad9bda7e98 WatchSource:0}: Error finding container 14c4c66fd605ed3fa4938573d9bb5de7481dcb3d9ac3050caea4a9ad9bda7e98: Status 404 returned error can't find the container with id 14c4c66fd605ed3fa4938573d9bb5de7481dcb3d9ac3050caea4a9ad9bda7e98 Jul 16 16:35:18.687289 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:18.687250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" event={"ID":"b36e52a4-3ae2-4b5f-b688-867bae24fb61","Type":"ContainerStarted","Data":"14c4c66fd605ed3fa4938573d9bb5de7481dcb3d9ac3050caea4a9ad9bda7e98"} Jul 16 16:35:19.693745 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:19.693707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" event={"ID":"b36e52a4-3ae2-4b5f-b688-867bae24fb61","Type":"ContainerStarted","Data":"25f7f5380d84f60cf87c941de81a690dc9e29f3d07bb87ae0205a4ae70953334"} Jul 16 16:35:19.714035 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:19.713981 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" podStartSLOduration=1.5632394600000001 podStartE2EDuration="2.713965508s" totalImagesPullingTime="1.150726048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:35:19.710737208 +0000 UTC m=+595.518849213" watchObservedRunningTime="2026-07-16 16:35:19.713965508 +0000 UTC m=+595.522077522" Jul 16 16:35:24.574012 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:24.573982 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:35:24.574450 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:24.574295 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:35:28.250082 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:28.250021 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:28.252467 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:28.250101 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:28.262867 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:28.262841 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:28.750085 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:28.750042 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xlgsp" Jul 16 16:35:32.031579 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.031537 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw"] Jul 16 16:35:32.036744 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.036718 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.039216 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.039186 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 16 16:35:32.042272 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.042246 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw"] Jul 16 16:35:32.105191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.105152 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7rcc\" (UniqueName: \"kubernetes.io/projected/84431927-83f7-414f-84d0-beceffb4b972-kube-api-access-l7rcc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.105361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.105214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.105361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.105258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.105361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.105287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.105361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.105347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.105502 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.105382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84431927-83f7-414f-84d0-beceffb4b972-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206425 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206425 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206617 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206603 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206683 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84431927-83f7-414f-84d0-beceffb4b972-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206720 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7rcc\" (UniqueName: \"kubernetes.io/projected/84431927-83f7-414f-84d0-beceffb4b972-kube-api-access-l7rcc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.206838 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206813 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.207020 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206844 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.207118 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.206955 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.209106 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.209084 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84431927-83f7-414f-84d0-beceffb4b972-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.209467 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.209445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84431927-83f7-414f-84d0-beceffb4b972-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.214853 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.214829 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7rcc\" (UniqueName: \"kubernetes.io/projected/84431927-83f7-414f-84d0-beceffb4b972-kube-api-access-l7rcc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw\" (UID: \"84431927-83f7-414f-84d0-beceffb4b972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.349220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.349136 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:32.483250 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.483220 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw"] Jul 16 16:35:32.485142 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:35:32.485116 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84431927_83f7_414f_84d0_beceffb4b972.slice/crio-bf96a30f80a84696a39162ab97b7683707111abb7cffe75a05450e3d7e80fb28 WatchSource:0}: Error finding container bf96a30f80a84696a39162ab97b7683707111abb7cffe75a05450e3d7e80fb28: Status 404 returned error can't find the container with id bf96a30f80a84696a39162ab97b7683707111abb7cffe75a05450e3d7e80fb28 Jul 16 16:35:32.486693 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.486668 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:35:32.759947 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.759892 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" event={"ID":"84431927-83f7-414f-84d0-beceffb4b972","Type":"ContainerStarted","Data":"594fda0a991dceff0fb3e4368e1d5fd1415e0e437beef846347e629f06d27b0c"} Jul 16 16:35:32.760156 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.759958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" event={"ID":"84431927-83f7-414f-84d0-beceffb4b972","Type":"ContainerStarted","Data":"bf96a30f80a84696a39162ab97b7683707111abb7cffe75a05450e3d7e80fb28"} Jul 16 16:35:32.781455 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:32.781405 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" podStartSLOduration=0.594524852 podStartE2EDuration="781.389389ms" totalImagesPullingTime="186.864537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:35:32.778796665 +0000 UTC m=+608.586908695" watchObservedRunningTime="2026-07-16 16:35:32.781389389 +0000 UTC m=+608.589501445" Jul 16 16:35:42.349784 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:42.349744 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:42.350229 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:42.349804 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:42.362396 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:42.362371 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:42.819966 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:42.819931 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw" Jul 16 16:35:53.763190 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.763114 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp"] Jul 16 16:35:53.769347 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.769326 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.771724 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.771702 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 16 16:35:53.779775 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.779750 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp"] Jul 16 16:35:53.894155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.894109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vgwd\" (UniqueName: \"kubernetes.io/projected/ec147ede-4d15-430c-9c29-89d221e1cdb2-kube-api-access-9vgwd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.894155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.894157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.894430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.894216 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.894430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.894281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.894430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.894295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec147ede-4d15-430c-9c29-89d221e1cdb2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.894430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.894327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995316 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec147ede-4d15-430c-9c29-89d221e1cdb2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995519 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995343 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995519 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vgwd\" (UniqueName: \"kubernetes.io/projected/ec147ede-4d15-430c-9c29-89d221e1cdb2-kube-api-access-9vgwd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995519 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995519 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995456 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995765 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995743 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995809 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995774 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.995845 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.995807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.997896 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.997874 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec147ede-4d15-430c-9c29-89d221e1cdb2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:53.998109 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:53.998093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec147ede-4d15-430c-9c29-89d221e1cdb2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:54.006017 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:54.005992 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vgwd\" (UniqueName: \"kubernetes.io/projected/ec147ede-4d15-430c-9c29-89d221e1cdb2-kube-api-access-9vgwd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp\" (UID: \"ec147ede-4d15-430c-9c29-89d221e1cdb2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:54.079787 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:54.079700 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:35:54.222611 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:54.222579 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp"] Jul 16 16:35:54.224835 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:35:54.224809 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec147ede_4d15_430c_9c29_89d221e1cdb2.slice/crio-cefc6f0d883ce2a65bda23ac0a9b3a13e1b3ab4923dc6798f331a4ec88e0ad99 WatchSource:0}: Error finding container cefc6f0d883ce2a65bda23ac0a9b3a13e1b3ab4923dc6798f331a4ec88e0ad99: Status 404 returned error can't find the container with id cefc6f0d883ce2a65bda23ac0a9b3a13e1b3ab4923dc6798f331a4ec88e0ad99 Jul 16 16:35:54.871604 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:54.871571 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" event={"ID":"ec147ede-4d15-430c-9c29-89d221e1cdb2","Type":"ContainerStarted","Data":"617aa6afee8a49c3dbadc3237ec541c1bbdd566ffb6fe4ade8fa344459a38583"} Jul 16 16:35:54.871967 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:54.871612 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" event={"ID":"ec147ede-4d15-430c-9c29-89d221e1cdb2","Type":"ContainerStarted","Data":"cefc6f0d883ce2a65bda23ac0a9b3a13e1b3ab4923dc6798f331a4ec88e0ad99"} Jul 16 16:35:54.900802 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:35:54.898132 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" podStartSLOduration=1.7128103810000002 podStartE2EDuration="1.898110619s" totalImagesPullingTime="185.300238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:35:54.894831957 +0000 UTC m=+630.702943970" watchObservedRunningTime="2026-07-16 16:35:54.898110619 +0000 UTC m=+630.706222634" Jul 16 16:36:04.080648 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:04.080614 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:36:04.081197 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:04.080695 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:36:04.093395 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:04.093369 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:36:04.937802 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:04.937776 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp" Jul 16 16:36:33.896667 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:33.896630 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56df85b9f9-zmsvx"] Jul 16 16:36:33.902086 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:33.901317 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:33.908169 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:33.908143 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56df85b9f9-zmsvx"] Jul 16 16:36:33.934455 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:33.934426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwnwd\" (UniqueName: \"kubernetes.io/projected/0a67dd93-cab8-425a-9ab5-bfe043ca1379-kube-api-access-jwnwd\") pod \"authorino-56df85b9f9-zmsvx\" (UID: \"0a67dd93-cab8-425a-9ab5-bfe043ca1379\") " pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:33.934613 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:33.934544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a67dd93-cab8-425a-9ab5-bfe043ca1379-tls-cert\") pod \"authorino-56df85b9f9-zmsvx\" (UID: \"0a67dd93-cab8-425a-9ab5-bfe043ca1379\") " pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:34.035812 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:34.035775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a67dd93-cab8-425a-9ab5-bfe043ca1379-tls-cert\") pod \"authorino-56df85b9f9-zmsvx\" (UID: \"0a67dd93-cab8-425a-9ab5-bfe043ca1379\") " pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:34.035982 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:34.035826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwnwd\" (UniqueName: \"kubernetes.io/projected/0a67dd93-cab8-425a-9ab5-bfe043ca1379-kube-api-access-jwnwd\") pod \"authorino-56df85b9f9-zmsvx\" (UID: \"0a67dd93-cab8-425a-9ab5-bfe043ca1379\") " pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:34.038364 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:34.038343 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a67dd93-cab8-425a-9ab5-bfe043ca1379-tls-cert\") pod \"authorino-56df85b9f9-zmsvx\" (UID: \"0a67dd93-cab8-425a-9ab5-bfe043ca1379\") " pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:34.045172 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:34.045145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwnwd\" (UniqueName: \"kubernetes.io/projected/0a67dd93-cab8-425a-9ab5-bfe043ca1379-kube-api-access-jwnwd\") pod \"authorino-56df85b9f9-zmsvx\" (UID: \"0a67dd93-cab8-425a-9ab5-bfe043ca1379\") " pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:34.213654 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:34.213611 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56df85b9f9-zmsvx" Jul 16 16:36:34.350245 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:34.350212 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56df85b9f9-zmsvx"] Jul 16 16:36:34.353208 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:36:34.353180 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a67dd93_cab8_425a_9ab5_bfe043ca1379.slice/crio-caa8ac5eef9165316b33ad14b908d044ca96a1835d12fdc21f825b4e96d2d4c4 WatchSource:0}: Error finding container caa8ac5eef9165316b33ad14b908d044ca96a1835d12fdc21f825b4e96d2d4c4: Status 404 returned error can't find the container with id caa8ac5eef9165316b33ad14b908d044ca96a1835d12fdc21f825b4e96d2d4c4 Jul 16 16:36:35.083811 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.083712 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56df85b9f9-zmsvx" event={"ID":"0a67dd93-cab8-425a-9ab5-bfe043ca1379","Type":"ContainerStarted","Data":"91f63918d6053fef1112758db2801df009a266e68f03334578f53f69f14ac58a"} Jul 16 16:36:35.083811 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.083777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56df85b9f9-zmsvx" event={"ID":"0a67dd93-cab8-425a-9ab5-bfe043ca1379","Type":"ContainerStarted","Data":"caa8ac5eef9165316b33ad14b908d044ca96a1835d12fdc21f825b4e96d2d4c4"} Jul 16 16:36:35.104592 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.104531 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56df85b9f9-zmsvx" podStartSLOduration=1.7575500480000001 podStartE2EDuration="2.104508104s" totalImagesPullingTime="346.958056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:36:35.099710046 +0000 UTC m=+670.907822060" watchObservedRunningTime="2026-07-16 16:36:35.104508104 +0000 UTC m=+670.912620120" Jul 16 16:36:35.152724 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.152693 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c675c59db-64c9m"] Jul 16 16:36:35.152913 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.152886 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c675c59db-64c9m" podUID="e206b7da-f819-4461-8255-559045af3a24" containerName="authorino" containerID="cri-o://a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8" gracePeriod=30 Jul 16 16:36:35.430669 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.430643 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:36:35.549992 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.549954 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjstn\" (UniqueName: \"kubernetes.io/projected/e206b7da-f819-4461-8255-559045af3a24-kube-api-access-gjstn\") pod \"e206b7da-f819-4461-8255-559045af3a24\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " Jul 16 16:36:35.549992 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.549987 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e206b7da-f819-4461-8255-559045af3a24-tls-cert\") pod \"e206b7da-f819-4461-8255-559045af3a24\" (UID: \"e206b7da-f819-4461-8255-559045af3a24\") " Jul 16 16:36:35.552361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.552325 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e206b7da-f819-4461-8255-559045af3a24-kube-api-access-gjstn" (OuterVolumeSpecName: "kube-api-access-gjstn") pod "e206b7da-f819-4461-8255-559045af3a24" (UID: "e206b7da-f819-4461-8255-559045af3a24"). InnerVolumeSpecName "kube-api-access-gjstn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:36:35.561792 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.561758 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e206b7da-f819-4461-8255-559045af3a24-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e206b7da-f819-4461-8255-559045af3a24" (UID: "e206b7da-f819-4461-8255-559045af3a24"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:36:35.650647 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.650562 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjstn\" (UniqueName: \"kubernetes.io/projected/e206b7da-f819-4461-8255-559045af3a24-kube-api-access-gjstn\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:36:35.650647 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:35.650591 2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e206b7da-f819-4461-8255-559045af3a24-tls-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:36:36.092199 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.092162 2560 generic.go:356] "Generic (PLEG): container finished" podID="e206b7da-f819-4461-8255-559045af3a24" containerID="a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8" exitCode=0 Jul 16 16:36:36.092680 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.092258 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c675c59db-64c9m" Jul 16 16:36:36.092680 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.092305 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c675c59db-64c9m" event={"ID":"e206b7da-f819-4461-8255-559045af3a24","Type":"ContainerDied","Data":"a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8"} Jul 16 16:36:36.092680 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.092345 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c675c59db-64c9m" event={"ID":"e206b7da-f819-4461-8255-559045af3a24","Type":"ContainerDied","Data":"35aa956f4a17bc277c8239101fd98eb4be79d49090fa3b24503a3856378de25d"} Jul 16 16:36:36.092680 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.092361 2560 scope.go:117] "RemoveContainer" containerID="a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8" Jul 16 16:36:36.102300 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.102274 2560 scope.go:117] "RemoveContainer" containerID="a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8" Jul 16 16:36:36.102640 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:36:36.102620 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8\": container with ID starting with a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8 not found: ID does not exist" containerID="a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8" Jul 16 16:36:36.102708 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.102650 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8"} err="failed to get container status \"a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8\": rpc error: code = NotFound desc = could not find container \"a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8\": container with ID starting with a44f4194bae6eb5657009409131688ee7c0ee02f7f3e2276b3ebc8cae4e4bfc8 not found: ID does not exist" Jul 16 16:36:36.149780 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.149745 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c675c59db-64c9m"] Jul 16 16:36:36.154408 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.154380 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c675c59db-64c9m"] Jul 16 16:36:36.657410 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:36:36.657364 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e206b7da-f819-4461-8255-559045af3a24" path="/var/lib/kubelet/pods/e206b7da-f819-4461-8255-559045af3a24/volumes" Jul 16 16:37:50.613589 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:50.613551 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-mj7vf"] Jul 16 16:37:50.614118 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:50.613779 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" podUID="198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" containerName="manager" containerID="cri-o://0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d" gracePeriod=30 Jul 16 16:37:50.865614 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:50.865551 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:37:51.004976 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.004948 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert\") pod \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " Jul 16 16:37:51.005189 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.005016 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxwnm\" (UniqueName: \"kubernetes.io/projected/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-kube-api-access-vxwnm\") pod \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\" (UID: \"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2\") " Jul 16 16:37:51.007297 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.007268 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" (UID: "198df1fa-8ccb-45a4-ad24-43ea5a9f13c2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:37:51.007389 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.007320 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-kube-api-access-vxwnm" (OuterVolumeSpecName: "kube-api-access-vxwnm") pod "198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" (UID: "198df1fa-8ccb-45a4-ad24-43ea5a9f13c2"). InnerVolumeSpecName "kube-api-access-vxwnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:37:51.106472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.106439 2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-webhook-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:37:51.106472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.106468 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxwnm\" (UniqueName: \"kubernetes.io/projected/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2-kube-api-access-vxwnm\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:37:51.491664 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.491626 2560 generic.go:356] "Generic (PLEG): container finished" podID="198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" containerID="0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d" exitCode=0 Jul 16 16:37:51.491860 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.491678 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" Jul 16 16:37:51.491860 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.491720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" event={"ID":"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2","Type":"ContainerDied","Data":"0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d"} Jul 16 16:37:51.491860 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.491767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847bf9b68-mj7vf" event={"ID":"198df1fa-8ccb-45a4-ad24-43ea5a9f13c2","Type":"ContainerDied","Data":"321a1cf312f591a8e9ae0808fa5bf07a5851c866d37556d5e614b1ba0b5108fe"} Jul 16 16:37:51.491860 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.491789 2560 scope.go:117] "RemoveContainer" containerID="0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d" Jul 16 16:37:51.501442 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.501420 2560 scope.go:117] "RemoveContainer" containerID="0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d" Jul 16 16:37:51.501684 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:37:51.501667 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d\": container with ID starting with 0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d not found: ID does not exist" containerID="0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d" Jul 16 16:37:51.501747 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.501690 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d"} err="failed to get container status \"0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d\": rpc error: code = NotFound desc = could not find container \"0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d\": container with ID starting with 0bed4ec692f8d8987d4df5d842d01144b538e55051895b26791738e8de292b1d not found: ID does not exist" Jul 16 16:37:51.512700 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.512678 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-mj7vf"] Jul 16 16:37:51.516370 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:51.516343 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-mj7vf"] Jul 16 16:37:52.380340 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380301 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6847bf9b68-kxdlt"] Jul 16 16:37:52.380698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380661 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" containerName="manager" Jul 16 16:37:52.380698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380676 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" containerName="manager" Jul 16 16:37:52.380698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380695 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e206b7da-f819-4461-8255-559045af3a24" containerName="authorino" Jul 16 16:37:52.380817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380701 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e206b7da-f819-4461-8255-559045af3a24" containerName="authorino" Jul 16 16:37:52.380817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380760 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e206b7da-f819-4461-8255-559045af3a24" containerName="authorino" Jul 16 16:37:52.380817 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.380770 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" containerName="manager" Jul 16 16:37:52.385182 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.385163 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.387420 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.387395 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 16:37:52.387963 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.387940 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dcbvg\"" Jul 16 16:37:52.388084 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.387940 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 16:37:52.392655 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.392342 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-kxdlt"] Jul 16 16:37:52.517868 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.517826 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb35eee2-eba0-42bb-97e7-90dfd9b170ac-webhook-cert\") pod \"maas-controller-6847bf9b68-kxdlt\" (UID: \"bb35eee2-eba0-42bb-97e7-90dfd9b170ac\") " pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.518081 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.517884 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wnk4\" (UniqueName: \"kubernetes.io/projected/bb35eee2-eba0-42bb-97e7-90dfd9b170ac-kube-api-access-4wnk4\") pod \"maas-controller-6847bf9b68-kxdlt\" (UID: \"bb35eee2-eba0-42bb-97e7-90dfd9b170ac\") " pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.618627 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.618594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb35eee2-eba0-42bb-97e7-90dfd9b170ac-webhook-cert\") pod \"maas-controller-6847bf9b68-kxdlt\" (UID: \"bb35eee2-eba0-42bb-97e7-90dfd9b170ac\") " pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.618815 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.618642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wnk4\" (UniqueName: \"kubernetes.io/projected/bb35eee2-eba0-42bb-97e7-90dfd9b170ac-kube-api-access-4wnk4\") pod \"maas-controller-6847bf9b68-kxdlt\" (UID: \"bb35eee2-eba0-42bb-97e7-90dfd9b170ac\") " pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.621211 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.621187 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb35eee2-eba0-42bb-97e7-90dfd9b170ac-webhook-cert\") pod \"maas-controller-6847bf9b68-kxdlt\" (UID: \"bb35eee2-eba0-42bb-97e7-90dfd9b170ac\") " pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.627125 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.627104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wnk4\" (UniqueName: \"kubernetes.io/projected/bb35eee2-eba0-42bb-97e7-90dfd9b170ac-kube-api-access-4wnk4\") pod \"maas-controller-6847bf9b68-kxdlt\" (UID: \"bb35eee2-eba0-42bb-97e7-90dfd9b170ac\") " pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.656109 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.656020 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="198df1fa-8ccb-45a4-ad24-43ea5a9f13c2" path="/var/lib/kubelet/pods/198df1fa-8ccb-45a4-ad24-43ea5a9f13c2/volumes" Jul 16 16:37:52.696260 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.696225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:37:52.832597 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:52.832573 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6847bf9b68-kxdlt"] Jul 16 16:37:52.834989 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:37:52.834949 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb35eee2_eba0_42bb_97e7_90dfd9b170ac.slice/crio-ec5c9c41490f2229519ee40188be169291ab45e0a89d4275bf597b0bb47d4fea WatchSource:0}: Error finding container ec5c9c41490f2229519ee40188be169291ab45e0a89d4275bf597b0bb47d4fea: Status 404 returned error can't find the container with id ec5c9c41490f2229519ee40188be169291ab45e0a89d4275bf597b0bb47d4fea Jul 16 16:37:53.505508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:53.505416 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" event={"ID":"bb35eee2-eba0-42bb-97e7-90dfd9b170ac","Type":"ContainerStarted","Data":"82b70e705bdd34ee53a1781bfcccd1f8de96552f5f51b9cb3f56bd2634fd5640"} Jul 16 16:37:53.505508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:53.505472 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" event={"ID":"bb35eee2-eba0-42bb-97e7-90dfd9b170ac","Type":"ContainerStarted","Data":"ec5c9c41490f2229519ee40188be169291ab45e0a89d4275bf597b0bb47d4fea"} Jul 16 16:37:53.505508 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:37:53.505495 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:38:04.518670 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:38:04.518634 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" Jul 16 16:38:04.538375 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:38:04.538322 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6847bf9b68-kxdlt" podStartSLOduration=12.144895627 podStartE2EDuration="12.53830503s" totalImagesPullingTime="393.409403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:37:53.540528615 +0000 UTC m=+749.348640629" watchObservedRunningTime="2026-07-16 16:38:04.53830503 +0000 UTC m=+760.346417044" Jul 16 16:40:24.609037 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:40:24.608950 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:40:24.609809 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:40:24.609789 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:45:24.642823 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:45:24.642794 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:45:24.645689 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:45:24.645668 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:50:20.439844 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.439794 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8"] Jul 16 16:50:20.444449 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.444424 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.447490 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.447469 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7a95ef74-serving-cert\"" Jul 16 16:50:20.447610 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.447469 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 16:50:20.454599 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.454575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8"] Jul 16 16:50:20.506558 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.506521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.506558 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.506564 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj5hc\" (UniqueName: \"kubernetes.io/projected/5224513d-d4f6-4029-844d-f65bc8e75df7-kube-api-access-dj5hc\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.506821 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.506656 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224513d-d4f6-4029-844d-f65bc8e75df7-openshift-service-ca\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.607973 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.607940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.608180 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.607984 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj5hc\" (UniqueName: \"kubernetes.io/projected/5224513d-d4f6-4029-844d-f65bc8e75df7-kube-api-access-dj5hc\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.608180 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:50:20.608104 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-serving-cert: secret "maas-api-e2e-ait-7a95ef74-serving-cert" not found Jul 16 16:50:20.608180 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.608124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224513d-d4f6-4029-844d-f65bc8e75df7-openshift-service-ca\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.608352 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:50:20.608184 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls podName:5224513d-d4f6-4029-844d-f65bc8e75df7 nodeName:}" failed. No retries permitted until 2026-07-16 16:50:21.108163543 +0000 UTC m=+1496.916275538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls") pod "maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" (UID: "5224513d-d4f6-4029-844d-f65bc8e75df7") : secret "maas-api-e2e-ait-7a95ef74-serving-cert" not found Jul 16 16:50:20.608814 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.608793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224513d-d4f6-4029-844d-f65bc8e75df7-openshift-service-ca\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:20.624329 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:20.624298 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj5hc\" (UniqueName: \"kubernetes.io/projected/5224513d-d4f6-4029-844d-f65bc8e75df7-kube-api-access-dj5hc\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:21.111283 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:21.111245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:21.114008 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:21.113975 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls\") pod \"maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:21.356569 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:21.356525 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:21.506052 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:21.506024 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8"] Jul 16 16:50:21.508282 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:50:21.508254 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5224513d_d4f6_4029_844d_f65bc8e75df7.slice/crio-c9ee05aae22b5d4a76ad0b553ffe2c825793d99520c1df93377a9983195894c9 WatchSource:0}: Error finding container c9ee05aae22b5d4a76ad0b553ffe2c825793d99520c1df93377a9983195894c9: Status 404 returned error can't find the container with id c9ee05aae22b5d4a76ad0b553ffe2c825793d99520c1df93377a9983195894c9 Jul 16 16:50:21.509633 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:21.509616 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:50:22.459721 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:22.459688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" event={"ID":"5224513d-d4f6-4029-844d-f65bc8e75df7","Type":"ContainerStarted","Data":"dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7"} Jul 16 16:50:22.459940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:22.459741 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" event={"ID":"5224513d-d4f6-4029-844d-f65bc8e75df7","Type":"ContainerStarted","Data":"c9ee05aae22b5d4a76ad0b553ffe2c825793d99520c1df93377a9983195894c9"} Jul 16 16:50:22.459940 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:22.459768 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:50:22.478790 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:22.478739 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" podStartSLOduration=1.7259131810000001 podStartE2EDuration="2.478724174s" totalImagesPullingTime="752.810993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:50:22.476454537 +0000 UTC m=+1498.284566552" watchObservedRunningTime="2026-07-16 16:50:22.478724174 +0000 UTC m=+1498.286836187" Jul 16 16:50:24.677583 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:24.677552 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:50:24.682490 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:24.682457 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:50:28.470907 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:50:28.470879 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:51:36.091361 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.091319 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8"] Jul 16 16:51:36.091829 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.091530 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" podUID="5224513d-d4f6-4029-844d-f65bc8e75df7" containerName="maas-api" containerID="cri-o://dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7" gracePeriod=30 Jul 16 16:51:36.337282 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.337257 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:51:36.478721 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.478685 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls\") pod \"5224513d-d4f6-4029-844d-f65bc8e75df7\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " Jul 16 16:51:36.478916 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.478755 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224513d-d4f6-4029-844d-f65bc8e75df7-openshift-service-ca\") pod \"5224513d-d4f6-4029-844d-f65bc8e75df7\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " Jul 16 16:51:36.478916 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.478787 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj5hc\" (UniqueName: \"kubernetes.io/projected/5224513d-d4f6-4029-844d-f65bc8e75df7-kube-api-access-dj5hc\") pod \"5224513d-d4f6-4029-844d-f65bc8e75df7\" (UID: \"5224513d-d4f6-4029-844d-f65bc8e75df7\") " Jul 16 16:51:36.479129 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.479104 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5224513d-d4f6-4029-844d-f65bc8e75df7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5224513d-d4f6-4029-844d-f65bc8e75df7" (UID: "5224513d-d4f6-4029-844d-f65bc8e75df7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:51:36.481049 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.481022 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5224513d-d4f6-4029-844d-f65bc8e75df7" (UID: "5224513d-d4f6-4029-844d-f65bc8e75df7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:51:36.481049 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.481036 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5224513d-d4f6-4029-844d-f65bc8e75df7-kube-api-access-dj5hc" (OuterVolumeSpecName: "kube-api-access-dj5hc") pod "5224513d-d4f6-4029-844d-f65bc8e75df7" (UID: "5224513d-d4f6-4029-844d-f65bc8e75df7"). InnerVolumeSpecName "kube-api-access-dj5hc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:51:36.579619 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.579582 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5224513d-d4f6-4029-844d-f65bc8e75df7-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:51:36.579619 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.579611 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5224513d-d4f6-4029-844d-f65bc8e75df7-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:51:36.579619 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.579622 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj5hc\" (UniqueName: \"kubernetes.io/projected/5224513d-d4f6-4029-844d-f65bc8e75df7-kube-api-access-dj5hc\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:51:36.849907 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.849805 2560 generic.go:356] "Generic (PLEG): container finished" podID="5224513d-d4f6-4029-844d-f65bc8e75df7" containerID="dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7" exitCode=0 Jul 16 16:51:36.849907 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.849852 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" Jul 16 16:51:36.850174 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.849847 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" event={"ID":"5224513d-d4f6-4029-844d-f65bc8e75df7","Type":"ContainerDied","Data":"dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7"} Jul 16 16:51:36.850174 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.849974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8" event={"ID":"5224513d-d4f6-4029-844d-f65bc8e75df7","Type":"ContainerDied","Data":"c9ee05aae22b5d4a76ad0b553ffe2c825793d99520c1df93377a9983195894c9"} Jul 16 16:51:36.850174 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.849998 2560 scope.go:117] "RemoveContainer" containerID="dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7" Jul 16 16:51:36.859325 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.859306 2560 scope.go:117] "RemoveContainer" containerID="dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7" Jul 16 16:51:36.859643 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:51:36.859624 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7\": container with ID starting with dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7 not found: ID does not exist" containerID="dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7" Jul 16 16:51:36.859693 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.859652 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7"} err="failed to get container status \"dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7\": rpc error: code = NotFound desc = could not find container \"dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7\": container with ID starting with dcf4fe4f58bcf2a74267441a731db70cde2173ce220c4ef0a85f30cef4f3fec7 not found: ID does not exist" Jul 16 16:51:36.866778 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.866754 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8"] Jul 16 16:51:36.870659 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:36.870637 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a95ef74-fc7c85587-llpc8"] Jul 16 16:51:38.655481 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:38.655445 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5224513d-d4f6-4029-844d-f65bc8e75df7" path="/var/lib/kubelet/pods/5224513d-d4f6-4029-844d-f65bc8e75df7/volumes" Jul 16 16:51:52.810040 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.810007 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk"] Jul 16 16:51:52.810518 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.810425 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5224513d-d4f6-4029-844d-f65bc8e75df7" containerName="maas-api" Jul 16 16:51:52.810518 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.810437 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5224513d-d4f6-4029-844d-f65bc8e75df7" containerName="maas-api" Jul 16 16:51:52.810518 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.810510 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5224513d-d4f6-4029-844d-f65bc8e75df7" containerName="maas-api" Jul 16 16:51:52.814995 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.814972 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.817831 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.817810 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c61ec96a-serving-cert\"" Jul 16 16:51:52.818042 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.817894 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 16:51:52.823010 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.822983 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7e997ac-429d-4b59-aa28-94e6fe9655af-openshift-service-ca\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.823121 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.823071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.823175 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.823134 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9gw9\" (UniqueName: \"kubernetes.io/projected/a7e997ac-429d-4b59-aa28-94e6fe9655af-kube-api-access-j9gw9\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.831104 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.831079 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk"] Jul 16 16:51:52.924053 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.924012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7e997ac-429d-4b59-aa28-94e6fe9655af-openshift-service-ca\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.924285 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.924092 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.924285 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.924129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gw9\" (UniqueName: \"kubernetes.io/projected/a7e997ac-429d-4b59-aa28-94e6fe9655af-kube-api-access-j9gw9\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.924411 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:51:52.924284 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-serving-cert: secret "maas-api-e2e-migrate-c61ec96a-serving-cert" not found Jul 16 16:51:52.924411 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:51:52.924376 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls podName:a7e997ac-429d-4b59-aa28-94e6fe9655af nodeName:}" failed. No retries permitted until 2026-07-16 16:51:53.424352985 +0000 UTC m=+1589.232464990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls") pod "maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" (UID: "a7e997ac-429d-4b59-aa28-94e6fe9655af") : secret "maas-api-e2e-migrate-c61ec96a-serving-cert" not found Jul 16 16:51:52.924668 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.924649 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7e997ac-429d-4b59-aa28-94e6fe9655af-openshift-service-ca\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:52.935411 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:52.935381 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9gw9\" (UniqueName: \"kubernetes.io/projected/a7e997ac-429d-4b59-aa28-94e6fe9655af-kube-api-access-j9gw9\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:53.427126 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:53.427088 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:53.429605 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:53.429587 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls\") pod \"maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:53.726302 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:53.726206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:53.877271 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:53.877247 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk"] Jul 16 16:51:53.879838 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:51:53.879807 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e997ac_429d_4b59_aa28_94e6fe9655af.slice/crio-25ff95dac99bfb78f0a376cef5183321bed0fd9d15f157871e35fc39b40349e6 WatchSource:0}: Error finding container 25ff95dac99bfb78f0a376cef5183321bed0fd9d15f157871e35fc39b40349e6: Status 404 returned error can't find the container with id 25ff95dac99bfb78f0a376cef5183321bed0fd9d15f157871e35fc39b40349e6 Jul 16 16:51:53.941267 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:53.941230 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" event={"ID":"a7e997ac-429d-4b59-aa28-94e6fe9655af","Type":"ContainerStarted","Data":"25ff95dac99bfb78f0a376cef5183321bed0fd9d15f157871e35fc39b40349e6"} Jul 16 16:51:54.948372 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:54.948341 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" event={"ID":"a7e997ac-429d-4b59-aa28-94e6fe9655af","Type":"ContainerStarted","Data":"ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001"} Jul 16 16:51:54.948738 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:54.948498 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:51:54.982556 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:51:54.982503 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" podStartSLOduration=2.468927076 podStartE2EDuration="2.9824864s" totalImagesPullingTime="513.559324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:51:54.98099152 +0000 UTC m=+1590.789103535" watchObservedRunningTime="2026-07-16 16:51:54.9824864 +0000 UTC m=+1590.790598413" Jul 16 16:52:00.958667 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:00.958635 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:52:57.906791 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:57.906758 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk"] Jul 16 16:52:57.907341 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:57.906972 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" podUID="a7e997ac-429d-4b59-aa28-94e6fe9655af" containerName="maas-api" containerID="cri-o://ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001" gracePeriod=30 Jul 16 16:52:58.155545 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.155521 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:52:58.180107 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.180083 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls\") pod \"a7e997ac-429d-4b59-aa28-94e6fe9655af\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " Jul 16 16:52:58.180254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.180197 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9gw9\" (UniqueName: \"kubernetes.io/projected/a7e997ac-429d-4b59-aa28-94e6fe9655af-kube-api-access-j9gw9\") pod \"a7e997ac-429d-4b59-aa28-94e6fe9655af\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " Jul 16 16:52:58.180317 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.180253 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7e997ac-429d-4b59-aa28-94e6fe9655af-openshift-service-ca\") pod \"a7e997ac-429d-4b59-aa28-94e6fe9655af\" (UID: \"a7e997ac-429d-4b59-aa28-94e6fe9655af\") " Jul 16 16:52:58.180604 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.180574 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7e997ac-429d-4b59-aa28-94e6fe9655af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7e997ac-429d-4b59-aa28-94e6fe9655af" (UID: "a7e997ac-429d-4b59-aa28-94e6fe9655af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:52:58.182352 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.182319 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7e997ac-429d-4b59-aa28-94e6fe9655af" (UID: "a7e997ac-429d-4b59-aa28-94e6fe9655af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:52:58.182456 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.182399 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e997ac-429d-4b59-aa28-94e6fe9655af-kube-api-access-j9gw9" (OuterVolumeSpecName: "kube-api-access-j9gw9") pod "a7e997ac-429d-4b59-aa28-94e6fe9655af" (UID: "a7e997ac-429d-4b59-aa28-94e6fe9655af"). InnerVolumeSpecName "kube-api-access-j9gw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:52:58.279093 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.279042 2560 generic.go:356] "Generic (PLEG): container finished" podID="a7e997ac-429d-4b59-aa28-94e6fe9655af" containerID="ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001" exitCode=0 Jul 16 16:52:58.279267 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.279103 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" Jul 16 16:52:58.279267 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.279129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" event={"ID":"a7e997ac-429d-4b59-aa28-94e6fe9655af","Type":"ContainerDied","Data":"ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001"} Jul 16 16:52:58.279267 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.279180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk" event={"ID":"a7e997ac-429d-4b59-aa28-94e6fe9655af","Type":"ContainerDied","Data":"25ff95dac99bfb78f0a376cef5183321bed0fd9d15f157871e35fc39b40349e6"} Jul 16 16:52:58.279267 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.279214 2560 scope.go:117] "RemoveContainer" containerID="ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001" Jul 16 16:52:58.280987 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.280965 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9gw9\" (UniqueName: \"kubernetes.io/projected/a7e997ac-429d-4b59-aa28-94e6fe9655af-kube-api-access-j9gw9\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:52:58.280987 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.280988 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7e997ac-429d-4b59-aa28-94e6fe9655af-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:52:58.281255 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.280998 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7e997ac-429d-4b59-aa28-94e6fe9655af-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:52:58.288907 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.288890 2560 scope.go:117] "RemoveContainer" containerID="ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001" Jul 16 16:52:58.289153 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:52:58.289135 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001\": container with ID starting with ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001 not found: ID does not exist" containerID="ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001" Jul 16 16:52:58.289206 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.289160 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001"} err="failed to get container status \"ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001\": rpc error: code = NotFound desc = could not find container \"ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001\": container with ID starting with ecc68ce461df010e1afe0ec062fa631f46ba0880cbd04990d7c059453fa01001 not found: ID does not exist" Jul 16 16:52:58.300093 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.300069 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk"] Jul 16 16:52:58.304812 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.304793 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c61ec96a-6494f48754-f2lfk"] Jul 16 16:52:58.656016 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:52:58.655921 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e997ac-429d-4b59-aa28-94e6fe9655af" path="/var/lib/kubelet/pods/a7e997ac-429d-4b59-aa28-94e6fe9655af/volumes" Jul 16 16:53:05.521122 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.521083 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b"] Jul 16 16:53:05.521601 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.521496 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e997ac-429d-4b59-aa28-94e6fe9655af" containerName="maas-api" Jul 16 16:53:05.521601 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.521509 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e997ac-429d-4b59-aa28-94e6fe9655af" containerName="maas-api" Jul 16 16:53:05.521601 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.521583 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e997ac-429d-4b59-aa28-94e6fe9655af" containerName="maas-api" Jul 16 16:53:05.525878 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.525856 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.528233 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.528213 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fb78c5b0-serving-cert\"" Jul 16 16:53:05.528356 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.528250 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 16:53:05.537670 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.537648 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b"] Jul 16 16:53:05.645080 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.645024 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-maas-api-tls\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.645252 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.645092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-openshift-service-ca\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.645252 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.645165 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ql65\" (UniqueName: \"kubernetes.io/projected/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-kube-api-access-2ql65\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.746131 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.746090 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-maas-api-tls\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.746320 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.746141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-openshift-service-ca\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.746320 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.746199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ql65\" (UniqueName: \"kubernetes.io/projected/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-kube-api-access-2ql65\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.746782 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.746762 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-openshift-service-ca\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.748638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.748615 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-maas-api-tls\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.763389 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.763366 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ql65\" (UniqueName: \"kubernetes.io/projected/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-kube-api-access-2ql65\") pod \"maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.836513 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.836415 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:05.982307 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:05.982276 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b"] Jul 16 16:53:05.984770 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:53:05.984742 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b5e2f4_606c_472a_a0dc_ec7e6847d2a9.slice/crio-d998bfd56d80a2fbc37110f852fe282779ab88220955a7363e9278a313c43963 WatchSource:0}: Error finding container d998bfd56d80a2fbc37110f852fe282779ab88220955a7363e9278a313c43963: Status 404 returned error can't find the container with id d998bfd56d80a2fbc37110f852fe282779ab88220955a7363e9278a313c43963 Jul 16 16:53:06.322135 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:06.322095 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" event={"ID":"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9","Type":"ContainerStarted","Data":"d998bfd56d80a2fbc37110f852fe282779ab88220955a7363e9278a313c43963"} Jul 16 16:53:07.328638 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:07.328596 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" event={"ID":"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9","Type":"ContainerStarted","Data":"0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da"} Jul 16 16:53:07.329148 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:07.328693 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:53:07.378689 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:07.378639 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" podStartSLOduration=1.856892116 podStartE2EDuration="2.378624799s" totalImagesPullingTime="521.732683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:53:07.37575551 +0000 UTC m=+1663.183867523" watchObservedRunningTime="2026-07-16 16:53:07.378624799 +0000 UTC m=+1663.186736817" Jul 16 16:53:13.339783 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:53:13.339755 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:54:21.160314 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.160262 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b"] Jul 16 16:54:21.160898 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.160521 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" podUID="a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" containerName="maas-api" containerID="cri-o://0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da" gracePeriod=30 Jul 16 16:54:21.415590 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.415526 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:54:21.605241 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.605205 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-openshift-service-ca\") pod \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " Jul 16 16:54:21.605424 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.605254 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-maas-api-tls\") pod \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " Jul 16 16:54:21.605424 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.605313 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ql65\" (UniqueName: \"kubernetes.io/projected/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-kube-api-access-2ql65\") pod \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\" (UID: \"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9\") " Jul 16 16:54:21.605596 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.605574 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" (UID: "a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:54:21.607603 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.607566 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-kube-api-access-2ql65" (OuterVolumeSpecName: "kube-api-access-2ql65") pod "a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" (UID: "a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9"). InnerVolumeSpecName "kube-api-access-2ql65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:54:21.607603 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.607589 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" (UID: "a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:54:21.706676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.706644 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:54:21.706676 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.706675 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:54:21.706873 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.706684 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ql65\" (UniqueName: \"kubernetes.io/projected/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9-kube-api-access-2ql65\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:54:21.729022 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.728986 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" containerID="0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da" exitCode=0 Jul 16 16:54:21.729195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.729048 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" Jul 16 16:54:21.729195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.729047 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" event={"ID":"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9","Type":"ContainerDied","Data":"0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da"} Jul 16 16:54:21.729195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.729136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b" event={"ID":"a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9","Type":"ContainerDied","Data":"d998bfd56d80a2fbc37110f852fe282779ab88220955a7363e9278a313c43963"} Jul 16 16:54:21.729195 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.729174 2560 scope.go:117] "RemoveContainer" containerID="0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da" Jul 16 16:54:21.745593 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.745576 2560 scope.go:117] "RemoveContainer" containerID="0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da" Jul 16 16:54:21.745853 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:54:21.745833 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da\": container with ID starting with 0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da not found: ID does not exist" containerID="0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da" Jul 16 16:54:21.745896 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.745863 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da"} err="failed to get container status \"0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da\": rpc error: code = NotFound desc = could not find container \"0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da\": container with ID starting with 0b0b71f364174501ba55aa609027a5bf5603cb680f7bb7ff12cab4931bbfb3da not found: ID does not exist" Jul 16 16:54:21.754784 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.754762 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b"] Jul 16 16:54:21.758877 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:21.758856 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fb78c5b0-67b8c9dbdd-hgf5b"] Jul 16 16:54:22.655229 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:22.655191 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" path="/var/lib/kubelet/pods/a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9/volumes" Jul 16 16:54:38.296536 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.296501 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl"] Jul 16 16:54:38.297006 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.296848 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" containerName="maas-api" Jul 16 16:54:38.297006 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.296858 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" containerName="maas-api" Jul 16 16:54:38.297006 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.296932 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b5e2f4-606c-472a-a0dc-ec7e6847d2a9" containerName="maas-api" Jul 16 16:54:38.301049 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.301020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.303335 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.303311 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ac435f2b-serving-cert\"" Jul 16 16:54:38.303472 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.303383 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 16:54:38.318749 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.318718 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl"] Jul 16 16:54:38.347547 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.347518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.347716 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.347551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t2lj\" (UniqueName: \"kubernetes.io/projected/dc755ef7-02c5-48c4-8559-ef809ffd1e79-kube-api-access-8t2lj\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.347716 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.347596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc755ef7-02c5-48c4-8559-ef809ffd1e79-openshift-service-ca\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.449052 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.449010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.449052 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.449082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t2lj\" (UniqueName: \"kubernetes.io/projected/dc755ef7-02c5-48c4-8559-ef809ffd1e79-kube-api-access-8t2lj\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.449342 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.449123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc755ef7-02c5-48c4-8559-ef809ffd1e79-openshift-service-ca\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.449342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:54:38.449153 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-serving-cert: secret "maas-api-e2e-derive-ac435f2b-serving-cert" not found Jul 16 16:54:38.449342 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:54:38.449223 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls podName:dc755ef7-02c5-48c4-8559-ef809ffd1e79 nodeName:}" failed. No retries permitted until 2026-07-16 16:54:38.949206702 +0000 UTC m=+1754.757318694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls") pod "maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" (UID: "dc755ef7-02c5-48c4-8559-ef809ffd1e79") : secret "maas-api-e2e-derive-ac435f2b-serving-cert" not found Jul 16 16:54:38.449770 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.449752 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc755ef7-02c5-48c4-8559-ef809ffd1e79-openshift-service-ca\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.465019 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.464998 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t2lj\" (UniqueName: \"kubernetes.io/projected/dc755ef7-02c5-48c4-8559-ef809ffd1e79-kube-api-access-8t2lj\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.953010 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.952972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:38.955575 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:38.955553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls\") pod \"maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:39.211976 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:39.211888 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:39.359196 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:39.359165 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl"] Jul 16 16:54:39.361248 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:54:39.361205 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc755ef7_02c5_48c4_8559_ef809ffd1e79.slice/crio-e63e51ad0d90af35a095ca877515e218391c25461ca6030adc8fc9f7fd8bc854 WatchSource:0}: Error finding container e63e51ad0d90af35a095ca877515e218391c25461ca6030adc8fc9f7fd8bc854: Status 404 returned error can't find the container with id e63e51ad0d90af35a095ca877515e218391c25461ca6030adc8fc9f7fd8bc854 Jul 16 16:54:39.823873 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:39.823836 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" event={"ID":"dc755ef7-02c5-48c4-8559-ef809ffd1e79","Type":"ContainerStarted","Data":"ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a"} Jul 16 16:54:39.824076 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:39.823900 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" event={"ID":"dc755ef7-02c5-48c4-8559-ef809ffd1e79","Type":"ContainerStarted","Data":"e63e51ad0d90af35a095ca877515e218391c25461ca6030adc8fc9f7fd8bc854"} Jul 16 16:54:39.824076 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:39.823968 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:54:39.840228 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:39.840172 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" podStartSLOduration=1.488755054 podStartE2EDuration="1.84015683s" totalImagesPullingTime="351.401776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:54:39.839717998 +0000 UTC m=+1755.647830012" watchObservedRunningTime="2026-07-16 16:54:39.84015683 +0000 UTC m=+1755.648268843" Jul 16 16:54:45.834771 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:54:45.834733 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:55:24.716594 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:24.716567 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:55:24.729408 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:24.729389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 16:55:53.212781 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.212748 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl"] Jul 16 16:55:53.213219 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.212943 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" podUID="dc755ef7-02c5-48c4-8559-ef809ffd1e79" containerName="maas-api" containerID="cri-o://ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a" gracePeriod=30 Jul 16 16:55:53.464007 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.463946 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:55:53.598798 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.598760 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc755ef7-02c5-48c4-8559-ef809ffd1e79-openshift-service-ca\") pod \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " Jul 16 16:55:53.598978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.598814 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t2lj\" (UniqueName: \"kubernetes.io/projected/dc755ef7-02c5-48c4-8559-ef809ffd1e79-kube-api-access-8t2lj\") pod \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " Jul 16 16:55:53.598978 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.598899 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls\") pod \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\" (UID: \"dc755ef7-02c5-48c4-8559-ef809ffd1e79\") " Jul 16 16:55:53.599220 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.599186 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc755ef7-02c5-48c4-8559-ef809ffd1e79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc755ef7-02c5-48c4-8559-ef809ffd1e79" (UID: "dc755ef7-02c5-48c4-8559-ef809ffd1e79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:55:53.601206 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.601178 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc755ef7-02c5-48c4-8559-ef809ffd1e79" (UID: "dc755ef7-02c5-48c4-8559-ef809ffd1e79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:55:53.601294 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.601182 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc755ef7-02c5-48c4-8559-ef809ffd1e79-kube-api-access-8t2lj" (OuterVolumeSpecName: "kube-api-access-8t2lj") pod "dc755ef7-02c5-48c4-8559-ef809ffd1e79" (UID: "dc755ef7-02c5-48c4-8559-ef809ffd1e79"). InnerVolumeSpecName "kube-api-access-8t2lj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:55:53.700240 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.700203 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc755ef7-02c5-48c4-8559-ef809ffd1e79-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:55:53.700240 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.700235 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc755ef7-02c5-48c4-8559-ef809ffd1e79-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:55:53.700240 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:53.700245 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t2lj\" (UniqueName: \"kubernetes.io/projected/dc755ef7-02c5-48c4-8559-ef809ffd1e79-kube-api-access-8t2lj\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:55:54.212005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.211970 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc755ef7-02c5-48c4-8559-ef809ffd1e79" containerID="ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a" exitCode=0 Jul 16 16:55:54.212201 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.212015 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" Jul 16 16:55:54.212201 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.212088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" event={"ID":"dc755ef7-02c5-48c4-8559-ef809ffd1e79","Type":"ContainerDied","Data":"ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a"} Jul 16 16:55:54.212201 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.212139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl" event={"ID":"dc755ef7-02c5-48c4-8559-ef809ffd1e79","Type":"ContainerDied","Data":"e63e51ad0d90af35a095ca877515e218391c25461ca6030adc8fc9f7fd8bc854"} Jul 16 16:55:54.212201 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.212162 2560 scope.go:117] "RemoveContainer" containerID="ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a" Jul 16 16:55:54.221937 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.221751 2560 scope.go:117] "RemoveContainer" containerID="ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a" Jul 16 16:55:54.222184 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:55:54.222030 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a\": container with ID starting with ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a not found: ID does not exist" containerID="ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a" Jul 16 16:55:54.222184 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.222072 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a"} err="failed to get container status \"ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a\": rpc error: code = NotFound desc = could not find container \"ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a\": container with ID starting with ba9ceee5aa423f8274cc98e3e94d2d69e64bbb9bfa6e1746507c95871030ad2a not found: ID does not exist" Jul 16 16:55:54.242480 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.242450 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl"] Jul 16 16:55:54.247085 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.247039 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ac435f2b-5f56477684-ft2bl"] Jul 16 16:55:54.656680 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:55:54.656596 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc755ef7-02c5-48c4-8559-ef809ffd1e79" path="/var/lib/kubelet/pods/dc755ef7-02c5-48c4-8559-ef809ffd1e79/volumes" Jul 16 16:57:49.805834 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.805801 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6"] Jul 16 16:57:49.806435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.806408 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc755ef7-02c5-48c4-8559-ef809ffd1e79" containerName="maas-api" Jul 16 16:57:49.806435 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.806431 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc755ef7-02c5-48c4-8559-ef809ffd1e79" containerName="maas-api" Jul 16 16:57:49.806598 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.806524 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc755ef7-02c5-48c4-8559-ef809ffd1e79" containerName="maas-api" Jul 16 16:57:49.810050 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.810021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.812155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.812135 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8a5e08ec-serving-cert\"" Jul 16 16:57:49.812276 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.812181 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 16:57:49.820752 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.820730 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6"] Jul 16 16:57:49.887407 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.887377 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005d39d0-4e7c-436d-89b5-7f474ad60d53-openshift-service-ca\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.887583 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.887433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.887583 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.887513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ptfr\" (UniqueName: \"kubernetes.io/projected/005d39d0-4e7c-436d-89b5-7f474ad60d53-kube-api-access-9ptfr\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.988754 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.988722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ptfr\" (UniqueName: \"kubernetes.io/projected/005d39d0-4e7c-436d-89b5-7f474ad60d53-kube-api-access-9ptfr\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.988918 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.988764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005d39d0-4e7c-436d-89b5-7f474ad60d53-openshift-service-ca\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.988918 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.988806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.988918 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:57:49.988897 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-serving-cert: secret "maas-api-e2e-mt-8a5e08ec-serving-cert" not found Jul 16 16:57:49.989037 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:57:49.988949 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls podName:005d39d0-4e7c-436d-89b5-7f474ad60d53 nodeName:}" failed. No retries permitted until 2026-07-16 16:57:50.488933876 +0000 UTC m=+1946.297045867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls") pod "maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" (UID: "005d39d0-4e7c-436d-89b5-7f474ad60d53") : secret "maas-api-e2e-mt-8a5e08ec-serving-cert" not found Jul 16 16:57:49.989430 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.989410 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005d39d0-4e7c-436d-89b5-7f474ad60d53-openshift-service-ca\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:49.997798 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:49.997768 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ptfr\" (UniqueName: \"kubernetes.io/projected/005d39d0-4e7c-436d-89b5-7f474ad60d53-kube-api-access-9ptfr\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:50.493491 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:50.493441 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:50.496043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:50.496013 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls\") pod \"maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:50.721892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:50.721858 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:50.859181 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:50.859157 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6"] Jul 16 16:57:50.860960 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:57:50.860926 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005d39d0_4e7c_436d_89b5_7f474ad60d53.slice/crio-66e1b26476a068f299149f77e2c0bb8312d0d24b910c6f9d00cb83b8ec2c307c WatchSource:0}: Error finding container 66e1b26476a068f299149f77e2c0bb8312d0d24b910c6f9d00cb83b8ec2c307c: Status 404 returned error can't find the container with id 66e1b26476a068f299149f77e2c0bb8312d0d24b910c6f9d00cb83b8ec2c307c Jul 16 16:57:50.862346 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:50.862329 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 16:57:51.819394 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:51.819362 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" event={"ID":"005d39d0-4e7c-436d-89b5-7f474ad60d53","Type":"ContainerStarted","Data":"66e1b26476a068f299149f77e2c0bb8312d0d24b910c6f9d00cb83b8ec2c307c"} Jul 16 16:57:53.832568 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:53.832538 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" event={"ID":"005d39d0-4e7c-436d-89b5-7f474ad60d53","Type":"ContainerStarted","Data":"c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815"} Jul 16 16:57:53.832942 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:53.832590 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:57:53.850336 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:53.850292 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" podStartSLOduration=2.700907252 podStartE2EDuration="4.850277233s" totalImagesPullingTime="2.149369981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:57:53.848574743 +0000 UTC m=+1949.656686758" watchObservedRunningTime="2026-07-16 16:57:53.850277233 +0000 UTC m=+1949.658389246" Jul 16 16:57:59.843713 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:57:59.843679 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:58:12.523043 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.523009 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6"] Jul 16 16:58:12.523535 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.523280 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" podUID="005d39d0-4e7c-436d-89b5-7f474ad60d53" containerName="maas-api" containerID="cri-o://c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815" gracePeriod=30 Jul 16 16:58:12.768389 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.768363 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:58:12.895030 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.894938 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005d39d0-4e7c-436d-89b5-7f474ad60d53-openshift-service-ca\") pod \"005d39d0-4e7c-436d-89b5-7f474ad60d53\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " Jul 16 16:58:12.895030 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.895016 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ptfr\" (UniqueName: \"kubernetes.io/projected/005d39d0-4e7c-436d-89b5-7f474ad60d53-kube-api-access-9ptfr\") pod \"005d39d0-4e7c-436d-89b5-7f474ad60d53\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " Jul 16 16:58:12.895277 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.895085 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls\") pod \"005d39d0-4e7c-436d-89b5-7f474ad60d53\" (UID: \"005d39d0-4e7c-436d-89b5-7f474ad60d53\") " Jul 16 16:58:12.895352 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.895328 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/005d39d0-4e7c-436d-89b5-7f474ad60d53-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "005d39d0-4e7c-436d-89b5-7f474ad60d53" (UID: "005d39d0-4e7c-436d-89b5-7f474ad60d53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:58:12.897256 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.897231 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/005d39d0-4e7c-436d-89b5-7f474ad60d53-kube-api-access-9ptfr" (OuterVolumeSpecName: "kube-api-access-9ptfr") pod "005d39d0-4e7c-436d-89b5-7f474ad60d53" (UID: "005d39d0-4e7c-436d-89b5-7f474ad60d53"). InnerVolumeSpecName "kube-api-access-9ptfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:58:12.897355 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.897274 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "005d39d0-4e7c-436d-89b5-7f474ad60d53" (UID: "005d39d0-4e7c-436d-89b5-7f474ad60d53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:58:12.930760 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.930729 2560 generic.go:356] "Generic (PLEG): container finished" podID="005d39d0-4e7c-436d-89b5-7f474ad60d53" containerID="c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815" exitCode=0 Jul 16 16:58:12.930865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.930769 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" event={"ID":"005d39d0-4e7c-436d-89b5-7f474ad60d53","Type":"ContainerDied","Data":"c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815"} Jul 16 16:58:12.930865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.930777 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" Jul 16 16:58:12.930865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.930822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6" event={"ID":"005d39d0-4e7c-436d-89b5-7f474ad60d53","Type":"ContainerDied","Data":"66e1b26476a068f299149f77e2c0bb8312d0d24b910c6f9d00cb83b8ec2c307c"} Jul 16 16:58:12.930865 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.930853 2560 scope.go:117] "RemoveContainer" containerID="c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815" Jul 16 16:58:12.940353 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.940335 2560 scope.go:117] "RemoveContainer" containerID="c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815" Jul 16 16:58:12.940609 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:58:12.940590 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815\": container with ID starting with c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815 not found: ID does not exist" containerID="c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815" Jul 16 16:58:12.940667 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.940622 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815"} err="failed to get container status \"c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815\": rpc error: code = NotFound desc = could not find container \"c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815\": container with ID starting with c14b1909ca5a5f28528fa34edc96d883817f8d3fa74753f7d28f90ed2daf3815 not found: ID does not exist" Jul 16 16:58:12.951492 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.951469 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6"] Jul 16 16:58:12.954785 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.954764 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a5e08ec-5bbdd7cf6f-kdjz6"] Jul 16 16:58:12.995698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.995670 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/005d39d0-4e7c-436d-89b5-7f474ad60d53-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:58:12.995698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.995692 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/005d39d0-4e7c-436d-89b5-7f474ad60d53-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:58:12.995698 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:12.995701 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ptfr\" (UniqueName: \"kubernetes.io/projected/005d39d0-4e7c-436d-89b5-7f474ad60d53-kube-api-access-9ptfr\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:58:14.655836 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:14.655798 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="005d39d0-4e7c-436d-89b5-7f474ad60d53" path="/var/lib/kubelet/pods/005d39d0-4e7c-436d-89b5-7f474ad60d53/volumes" Jul 16 16:58:51.247297 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.247023 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q"] Jul 16 16:58:51.247892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.247380 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="005d39d0-4e7c-436d-89b5-7f474ad60d53" containerName="maas-api" Jul 16 16:58:51.247892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.247392 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="005d39d0-4e7c-436d-89b5-7f474ad60d53" containerName="maas-api" Jul 16 16:58:51.247892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.247454 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="005d39d0-4e7c-436d-89b5-7f474ad60d53" containerName="maas-api" Jul 16 16:58:51.250581 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.250561 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.252623 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.252604 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-187946be-serving-cert\"" Jul 16 16:58:51.252812 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.252798 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 16:58:51.264005 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.263983 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q"] Jul 16 16:58:51.299895 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.299855 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-openshift-service-ca\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.300191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.299922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-maas-api-tls\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.300191 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.300139 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl6hb\" (UniqueName: \"kubernetes.io/projected/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-kube-api-access-rl6hb\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.401382 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.401347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl6hb\" (UniqueName: \"kubernetes.io/projected/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-kube-api-access-rl6hb\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.401578 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.401410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-openshift-service-ca\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.401578 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.401457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-maas-api-tls\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.402181 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.402161 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-openshift-service-ca\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.404014 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.403996 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-maas-api-tls\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.413197 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.413170 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl6hb\" (UniqueName: \"kubernetes.io/projected/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-kube-api-access-rl6hb\") pod \"maas-api-e2e-mt-187946be-c7fb7675d-5926q\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.561241 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.561153 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:51.704425 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:51.703778 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q"] Jul 16 16:58:51.708355 ip-10-0-131-94 kubenswrapper[2560]: W0716 16:58:51.708324 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30bce20e_7483_4706_bfd9_1cf87fdf0ed9.slice/crio-780d206af935381376d5651ce7893dcba7b3991596e5a3000dad9d394bc837d9 WatchSource:0}: Error finding container 780d206af935381376d5651ce7893dcba7b3991596e5a3000dad9d394bc837d9: Status 404 returned error can't find the container with id 780d206af935381376d5651ce7893dcba7b3991596e5a3000dad9d394bc837d9 Jul 16 16:58:52.136262 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:52.136225 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" event={"ID":"30bce20e-7483-4706-bfd9-1cf87fdf0ed9","Type":"ContainerStarted","Data":"780d206af935381376d5651ce7893dcba7b3991596e5a3000dad9d394bc837d9"} Jul 16 16:58:54.148283 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:54.148180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" event={"ID":"30bce20e-7483-4706-bfd9-1cf87fdf0ed9","Type":"ContainerStarted","Data":"3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886"} Jul 16 16:58:54.148724 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:54.148315 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:58:54.166155 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:58:54.166106 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" podStartSLOduration=1.164769521 podStartE2EDuration="3.166089732s" totalImagesPullingTime="2.001320211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:58:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:58:54.164510224 +0000 UTC m=+2009.972622238" watchObservedRunningTime="2026-07-16 16:58:54.166089732 +0000 UTC m=+2009.974201746" Jul 16 16:59:00.159121 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:00.159092 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:59:49.798743 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:49.798659 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q"] Jul 16 16:59:49.799351 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:49.798893 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" podUID="30bce20e-7483-4706-bfd9-1cf87fdf0ed9" containerName="maas-api" containerID="cri-o://3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886" gracePeriod=30 Jul 16 16:59:50.051621 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.051564 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:59:50.130298 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.130272 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-maas-api-tls\") pod \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " Jul 16 16:59:50.130478 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.130358 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-openshift-service-ca\") pod \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " Jul 16 16:59:50.130478 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.130378 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rl6hb\" (UniqueName: \"kubernetes.io/projected/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-kube-api-access-rl6hb\") pod \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\" (UID: \"30bce20e-7483-4706-bfd9-1cf87fdf0ed9\") " Jul 16 16:59:50.130763 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.130729 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30bce20e-7483-4706-bfd9-1cf87fdf0ed9" (UID: "30bce20e-7483-4706-bfd9-1cf87fdf0ed9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 16:59:50.132663 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.132637 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30bce20e-7483-4706-bfd9-1cf87fdf0ed9" (UID: "30bce20e-7483-4706-bfd9-1cf87fdf0ed9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:59:50.132764 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.132675 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-kube-api-access-rl6hb" (OuterVolumeSpecName: "kube-api-access-rl6hb") pod "30bce20e-7483-4706-bfd9-1cf87fdf0ed9" (UID: "30bce20e-7483-4706-bfd9-1cf87fdf0ed9"). InnerVolumeSpecName "kube-api-access-rl6hb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:59:50.231259 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.231228 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:59:50.231259 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.231254 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl6hb\" (UniqueName: \"kubernetes.io/projected/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-kube-api-access-rl6hb\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:59:50.231259 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.231264 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30bce20e-7483-4706-bfd9-1cf87fdf0ed9-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 16:59:50.445080 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.445026 2560 generic.go:356] "Generic (PLEG): container finished" podID="30bce20e-7483-4706-bfd9-1cf87fdf0ed9" containerID="3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886" exitCode=0 Jul 16 16:59:50.445254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.445090 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" Jul 16 16:59:50.445254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.445093 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" event={"ID":"30bce20e-7483-4706-bfd9-1cf87fdf0ed9","Type":"ContainerDied","Data":"3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886"} Jul 16 16:59:50.445254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.445146 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q" event={"ID":"30bce20e-7483-4706-bfd9-1cf87fdf0ed9","Type":"ContainerDied","Data":"780d206af935381376d5651ce7893dcba7b3991596e5a3000dad9d394bc837d9"} Jul 16 16:59:50.445254 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.445168 2560 scope.go:117] "RemoveContainer" containerID="3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886" Jul 16 16:59:50.454578 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.454561 2560 scope.go:117] "RemoveContainer" containerID="3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886" Jul 16 16:59:50.454836 ip-10-0-131-94 kubenswrapper[2560]: E0716 16:59:50.454817 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886\": container with ID starting with 3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886 not found: ID does not exist" containerID="3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886" Jul 16 16:59:50.454885 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.454845 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886"} err="failed to get container status \"3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886\": rpc error: code = NotFound desc = could not find container \"3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886\": container with ID starting with 3138318ca1a4b4221181ec25ddee13d8f5c10c888cc9a2814ab4c282d75ef886 not found: ID does not exist" Jul 16 16:59:50.467892 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.467872 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q"] Jul 16 16:59:50.472387 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.472365 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-187946be-c7fb7675d-5926q"] Jul 16 16:59:50.656151 ip-10-0-131-94 kubenswrapper[2560]: I0716 16:59:50.656122 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30bce20e-7483-4706-bfd9-1cf87fdf0ed9" path="/var/lib/kubelet/pods/30bce20e-7483-4706-bfd9-1cf87fdf0ed9/volumes" Jul 16 17:00:24.749077 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:00:24.749031 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:00:24.762522 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:00:24.762503 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:01:46.229839 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.229752 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g"] Jul 16 17:01:46.230308 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.230119 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30bce20e-7483-4706-bfd9-1cf87fdf0ed9" containerName="maas-api" Jul 16 17:01:46.230308 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.230131 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="30bce20e-7483-4706-bfd9-1cf87fdf0ed9" containerName="maas-api" Jul 16 17:01:46.230308 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.230197 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="30bce20e-7483-4706-bfd9-1cf87fdf0ed9" containerName="maas-api" Jul 16 17:01:46.233312 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.233293 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.236998 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.236977 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fa0ccd-serving-cert\"" Jul 16 17:01:46.237127 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.237106 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 17:01:46.245036 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.245012 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g"] Jul 16 17:01:46.327668 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.327631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpmrf\" (UniqueName: \"kubernetes.io/projected/cda58bc5-74e4-4ab4-9501-160f26bcd40d-kube-api-access-lpmrf\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.327846 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.327713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda58bc5-74e4-4ab4-9501-160f26bcd40d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.327846 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.327746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.428896 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.428862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpmrf\" (UniqueName: \"kubernetes.io/projected/cda58bc5-74e4-4ab4-9501-160f26bcd40d-kube-api-access-lpmrf\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.429077 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.428941 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda58bc5-74e4-4ab4-9501-160f26bcd40d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.429077 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.428989 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.429167 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:01:46.429116 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-serving-cert: secret "maas-api-e2e-inf-a-fa0ccd-serving-cert" not found Jul 16 17:01:46.429203 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:01:46.429188 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls podName:cda58bc5-74e4-4ab4-9501-160f26bcd40d nodeName:}" failed. No retries permitted until 2026-07-16 17:01:46.929169372 +0000 UTC m=+2182.737281371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls") pod "maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" (UID: "cda58bc5-74e4-4ab4-9501-160f26bcd40d") : secret "maas-api-e2e-inf-a-fa0ccd-serving-cert" not found Jul 16 17:01:46.429571 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.429552 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda58bc5-74e4-4ab4-9501-160f26bcd40d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.454611 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.454579 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpmrf\" (UniqueName: \"kubernetes.io/projected/cda58bc5-74e4-4ab4-9501-160f26bcd40d-kube-api-access-lpmrf\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.932919 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.932881 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:46.935482 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:46.935454 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:47.143861 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:47.143823 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:47.283188 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:47.283158 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g"] Jul 16 17:01:47.285100 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:01:47.285053 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda58bc5_74e4_4ab4_9501_160f26bcd40d.slice/crio-714d382617ca6316cec1cd18d68a90b4b08e03dab951f2e8f0252c0cf7b36545 WatchSource:0}: Error finding container 714d382617ca6316cec1cd18d68a90b4b08e03dab951f2e8f0252c0cf7b36545: Status 404 returned error can't find the container with id 714d382617ca6316cec1cd18d68a90b4b08e03dab951f2e8f0252c0cf7b36545 Jul 16 17:01:48.050403 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:48.050373 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" event={"ID":"cda58bc5-74e4-4ab4-9501-160f26bcd40d","Type":"ContainerStarted","Data":"8dff2ec82cede251b78116ea12b19e948c265938ac2ba93843524918aa56e364"} Jul 16 17:01:48.050571 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:48.050427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" event={"ID":"cda58bc5-74e4-4ab4-9501-160f26bcd40d","Type":"ContainerStarted","Data":"714d382617ca6316cec1cd18d68a90b4b08e03dab951f2e8f0252c0cf7b36545"} Jul 16 17:01:48.050571 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:48.050484 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:48.069381 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:48.069335 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" podStartSLOduration=1.45658719 podStartE2EDuration="2.069320138s" totalImagesPullingTime="612.732948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:01:48.068030131 +0000 UTC m=+2183.876142145" watchObservedRunningTime="2026-07-16 17:01:48.069320138 +0000 UTC m=+2183.877432152" Jul 16 17:01:54.061191 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.061160 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:01:54.710769 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.710733 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd"] Jul 16 17:01:54.714383 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.714359 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.716804 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.716779 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f37c57-serving-cert\"" Jul 16 17:01:54.729216 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.729191 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd"] Jul 16 17:01:54.803654 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.803621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q8vw\" (UniqueName: \"kubernetes.io/projected/2359ffd8-772a-4184-a040-eb45a07d26c5-kube-api-access-4q8vw\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.803826 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.803659 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.803826 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.803679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2359ffd8-772a-4184-a040-eb45a07d26c5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.904232 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.904198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q8vw\" (UniqueName: \"kubernetes.io/projected/2359ffd8-772a-4184-a040-eb45a07d26c5-kube-api-access-4q8vw\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.904232 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.904238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.904451 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.904257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2359ffd8-772a-4184-a040-eb45a07d26c5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.904451 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:01:54.904365 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-serving-cert: secret "maas-api-e2e-inf-b-f37c57-serving-cert" not found Jul 16 17:01:54.904451 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:01:54.904436 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls podName:2359ffd8-772a-4184-a040-eb45a07d26c5 nodeName:}" failed. No retries permitted until 2026-07-16 17:01:55.404415119 +0000 UTC m=+2191.212527113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls") pod "maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" (UID: "2359ffd8-772a-4184-a040-eb45a07d26c5") : secret "maas-api-e2e-inf-b-f37c57-serving-cert" not found Jul 16 17:01:54.904944 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.904925 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2359ffd8-772a-4184-a040-eb45a07d26c5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:54.914545 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:54.914518 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q8vw\" (UniqueName: \"kubernetes.io/projected/2359ffd8-772a-4184-a040-eb45a07d26c5-kube-api-access-4q8vw\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:55.408305 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:55.408267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:55.410767 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:55.410748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls\") pod \"maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:55.626882 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:55.626851 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:55.773475 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:55.773450 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd"] Jul 16 17:01:55.775433 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:01:55.775406 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2359ffd8_772a_4184_a040_eb45a07d26c5.slice/crio-02af51b0503ad01a193d5bc6c5c27209d42f16581aa738418a8ca648a1d2d5c5 WatchSource:0}: Error finding container 02af51b0503ad01a193d5bc6c5c27209d42f16581aa738418a8ca648a1d2d5c5: Status 404 returned error can't find the container with id 02af51b0503ad01a193d5bc6c5c27209d42f16581aa738418a8ca648a1d2d5c5 Jul 16 17:01:56.094525 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:56.094427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" event={"ID":"2359ffd8-772a-4184-a040-eb45a07d26c5","Type":"ContainerStarted","Data":"02af51b0503ad01a193d5bc6c5c27209d42f16581aa738418a8ca648a1d2d5c5"} Jul 16 17:01:57.102382 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:57.102343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" event={"ID":"2359ffd8-772a-4184-a040-eb45a07d26c5","Type":"ContainerStarted","Data":"ae06a023b611a718b89758608129dad8ae032165b18ade2b3c7b257c5ada6f38"} Jul 16 17:01:57.102790 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:57.102491 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:01:57.120666 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:01:57.120621 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" podStartSLOduration=2.753893109 podStartE2EDuration="3.120608046s" totalImagesPullingTime="366.714937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:01:57.118021759 +0000 UTC m=+2192.926133773" watchObservedRunningTime="2026-07-16 17:01:57.120608046 +0000 UTC m=+2192.928720101" Jul 16 17:02:03.113468 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:02:03.113438 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:03:40.424387 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.424352 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g"] Jul 16 17:03:40.424956 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.424584 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" podUID="cda58bc5-74e4-4ab4-9501-160f26bcd40d" containerName="maas-api" containerID="cri-o://8dff2ec82cede251b78116ea12b19e948c265938ac2ba93843524918aa56e364" gracePeriod=30 Jul 16 17:03:40.664493 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.664461 2560 generic.go:356] "Generic (PLEG): container finished" podID="cda58bc5-74e4-4ab4-9501-160f26bcd40d" containerID="8dff2ec82cede251b78116ea12b19e948c265938ac2ba93843524918aa56e364" exitCode=0 Jul 16 17:03:40.664622 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.664504 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" event={"ID":"cda58bc5-74e4-4ab4-9501-160f26bcd40d","Type":"ContainerDied","Data":"8dff2ec82cede251b78116ea12b19e948c265938ac2ba93843524918aa56e364"} Jul 16 17:03:40.664622 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.664547 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" event={"ID":"cda58bc5-74e4-4ab4-9501-160f26bcd40d","Type":"ContainerDied","Data":"714d382617ca6316cec1cd18d68a90b4b08e03dab951f2e8f0252c0cf7b36545"} Jul 16 17:03:40.664622 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.664566 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="714d382617ca6316cec1cd18d68a90b4b08e03dab951f2e8f0252c0cf7b36545" Jul 16 17:03:40.672144 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.672116 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:03:40.841929 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.841813 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda58bc5-74e4-4ab4-9501-160f26bcd40d-openshift-service-ca\") pod \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " Jul 16 17:03:40.841929 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.841880 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls\") pod \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " Jul 16 17:03:40.841929 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.841909 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpmrf\" (UniqueName: \"kubernetes.io/projected/cda58bc5-74e4-4ab4-9501-160f26bcd40d-kube-api-access-lpmrf\") pod \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\" (UID: \"cda58bc5-74e4-4ab4-9501-160f26bcd40d\") " Jul 16 17:03:40.842307 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.842278 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cda58bc5-74e4-4ab4-9501-160f26bcd40d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cda58bc5-74e4-4ab4-9501-160f26bcd40d" (UID: "cda58bc5-74e4-4ab4-9501-160f26bcd40d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:03:40.844165 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.844136 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cda58bc5-74e4-4ab4-9501-160f26bcd40d" (UID: "cda58bc5-74e4-4ab4-9501-160f26bcd40d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:03:40.844270 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.844210 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cda58bc5-74e4-4ab4-9501-160f26bcd40d-kube-api-access-lpmrf" (OuterVolumeSpecName: "kube-api-access-lpmrf") pod "cda58bc5-74e4-4ab4-9501-160f26bcd40d" (UID: "cda58bc5-74e4-4ab4-9501-160f26bcd40d"). InnerVolumeSpecName "kube-api-access-lpmrf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:03:40.942968 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.942929 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda58bc5-74e4-4ab4-9501-160f26bcd40d-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:03:40.942968 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.942970 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda58bc5-74e4-4ab4-9501-160f26bcd40d-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:03:40.943212 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:40.942985 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpmrf\" (UniqueName: \"kubernetes.io/projected/cda58bc5-74e4-4ab4-9501-160f26bcd40d-kube-api-access-lpmrf\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:03:41.670256 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:41.670224 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g" Jul 16 17:03:41.692505 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:41.692474 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g"] Jul 16 17:03:41.700708 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:41.700676 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa0ccd-5b9dd54bf9-t4b8g"] Jul 16 17:03:42.656863 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:03:42.656832 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cda58bc5-74e4-4ab4-9501-160f26bcd40d" path="/var/lib/kubelet/pods/cda58bc5-74e4-4ab4-9501-160f26bcd40d/volumes" Jul 16 17:04:06.673092 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:06.673037 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd"] Jul 16 17:04:06.673511 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:06.673334 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" podUID="2359ffd8-772a-4184-a040-eb45a07d26c5" containerName="maas-api" containerID="cri-o://ae06a023b611a718b89758608129dad8ae032165b18ade2b3c7b257c5ada6f38" gracePeriod=30 Jul 16 17:04:06.809653 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:06.809619 2560 generic.go:356] "Generic (PLEG): container finished" podID="2359ffd8-772a-4184-a040-eb45a07d26c5" containerID="ae06a023b611a718b89758608129dad8ae032165b18ade2b3c7b257c5ada6f38" exitCode=0 Jul 16 17:04:06.809818 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:06.809694 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" event={"ID":"2359ffd8-772a-4184-a040-eb45a07d26c5","Type":"ContainerDied","Data":"ae06a023b611a718b89758608129dad8ae032165b18ade2b3c7b257c5ada6f38"} Jul 16 17:04:06.931709 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:06.931647 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:04:07.072971 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.072914 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q8vw\" (UniqueName: \"kubernetes.io/projected/2359ffd8-772a-4184-a040-eb45a07d26c5-kube-api-access-4q8vw\") pod \"2359ffd8-772a-4184-a040-eb45a07d26c5\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " Jul 16 17:04:07.072971 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.072975 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2359ffd8-772a-4184-a040-eb45a07d26c5-openshift-service-ca\") pod \"2359ffd8-772a-4184-a040-eb45a07d26c5\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " Jul 16 17:04:07.073251 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.073009 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls\") pod \"2359ffd8-772a-4184-a040-eb45a07d26c5\" (UID: \"2359ffd8-772a-4184-a040-eb45a07d26c5\") " Jul 16 17:04:07.073486 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.073453 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2359ffd8-772a-4184-a040-eb45a07d26c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2359ffd8-772a-4184-a040-eb45a07d26c5" (UID: "2359ffd8-772a-4184-a040-eb45a07d26c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:04:07.075467 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.075429 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2359ffd8-772a-4184-a040-eb45a07d26c5" (UID: "2359ffd8-772a-4184-a040-eb45a07d26c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:04:07.075570 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.075485 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2359ffd8-772a-4184-a040-eb45a07d26c5-kube-api-access-4q8vw" (OuterVolumeSpecName: "kube-api-access-4q8vw") pod "2359ffd8-772a-4184-a040-eb45a07d26c5" (UID: "2359ffd8-772a-4184-a040-eb45a07d26c5"). InnerVolumeSpecName "kube-api-access-4q8vw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:04:07.173908 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.173866 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q8vw\" (UniqueName: \"kubernetes.io/projected/2359ffd8-772a-4184-a040-eb45a07d26c5-kube-api-access-4q8vw\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:04:07.173908 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.173903 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2359ffd8-772a-4184-a040-eb45a07d26c5-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:04:07.173908 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.173912 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2359ffd8-772a-4184-a040-eb45a07d26c5-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:04:07.816199 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.816174 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" Jul 16 17:04:07.816619 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.816164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd" event={"ID":"2359ffd8-772a-4184-a040-eb45a07d26c5","Type":"ContainerDied","Data":"02af51b0503ad01a193d5bc6c5c27209d42f16581aa738418a8ca648a1d2d5c5"} Jul 16 17:04:07.816619 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.816319 2560 scope.go:117] "RemoveContainer" containerID="ae06a023b611a718b89758608129dad8ae032165b18ade2b3c7b257c5ada6f38" Jul 16 17:04:07.839690 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.839649 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd"] Jul 16 17:04:07.852639 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:07.852613 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f37c57-75ccc8df58-clhvd"] Jul 16 17:04:08.656662 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:08.656623 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2359ffd8-772a-4184-a040-eb45a07d26c5" path="/var/lib/kubelet/pods/2359ffd8-772a-4184-a040-eb45a07d26c5/volumes" Jul 16 17:04:26.245649 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.245614 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr"] Jul 16 17:04:26.246042 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.246003 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2359ffd8-772a-4184-a040-eb45a07d26c5" containerName="maas-api" Jul 16 17:04:26.246042 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.246013 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2359ffd8-772a-4184-a040-eb45a07d26c5" containerName="maas-api" Jul 16 17:04:26.246042 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.246026 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cda58bc5-74e4-4ab4-9501-160f26bcd40d" containerName="maas-api" Jul 16 17:04:26.246042 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.246032 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cda58bc5-74e4-4ab4-9501-160f26bcd40d" containerName="maas-api" Jul 16 17:04:26.246197 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.246120 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2359ffd8-772a-4184-a040-eb45a07d26c5" containerName="maas-api" Jul 16 17:04:26.246197 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.246132 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cda58bc5-74e4-4ab4-9501-160f26bcd40d" containerName="maas-api" Jul 16 17:04:26.250600 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.250580 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.252579 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.252557 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 17:04:26.252780 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.252766 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6f111d-serving-cert\"" Jul 16 17:04:26.262576 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.262550 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr"] Jul 16 17:04:26.345086 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.345013 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.345292 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.345105 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gxl4\" (UniqueName: \"kubernetes.io/projected/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-kube-api-access-6gxl4\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.345292 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.345153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.446362 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.446318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.446572 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.446383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gxl4\" (UniqueName: \"kubernetes.io/projected/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-kube-api-access-6gxl4\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.446572 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.446435 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.446702 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:04:26.446572 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-serving-cert: secret "maas-api-e2e-api-a-6f111d-serving-cert" not found Jul 16 17:04:26.446702 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:04:26.446644 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls podName:f66eade8-0fea-4dde-9dac-bb338ff1d9a4 nodeName:}" failed. No retries permitted until 2026-07-16 17:04:26.946621188 +0000 UTC m=+2342.754733201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls") pod "maas-api-e2e-api-a-6f111d-587994488b-jwkdr" (UID: "f66eade8-0fea-4dde-9dac-bb338ff1d9a4") : secret "maas-api-e2e-api-a-6f111d-serving-cert" not found Jul 16 17:04:26.446964 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.446939 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-openshift-service-ca\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.455914 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.455878 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gxl4\" (UniqueName: \"kubernetes.io/projected/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-kube-api-access-6gxl4\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.951624 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.951584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:26.954296 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:26.954267 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls\") pod \"maas-api-e2e-api-a-6f111d-587994488b-jwkdr\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:27.160901 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.160867 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:27.310779 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:04:27.310751 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66eade8_0fea_4dde_9dac_bb338ff1d9a4.slice/crio-c47709d035103af985ab3da15ed2ac5df25c4e0fde4e0d4a8867d082521d790c WatchSource:0}: Error finding container c47709d035103af985ab3da15ed2ac5df25c4e0fde4e0d4a8867d082521d790c: Status 404 returned error can't find the container with id c47709d035103af985ab3da15ed2ac5df25c4e0fde4e0d4a8867d082521d790c Jul 16 17:04:27.311992 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.311975 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:04:27.312092 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.312026 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr"] Jul 16 17:04:27.929075 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.929029 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" event={"ID":"f66eade8-0fea-4dde-9dac-bb338ff1d9a4","Type":"ContainerStarted","Data":"63139b2c88270a9fff497c90f851dd2e2446ef598e0e831dae6a1fcc19ed1426"} Jul 16 17:04:27.929276 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.929110 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" event={"ID":"f66eade8-0fea-4dde-9dac-bb338ff1d9a4","Type":"ContainerStarted","Data":"c47709d035103af985ab3da15ed2ac5df25c4e0fde4e0d4a8867d082521d790c"} Jul 16 17:04:27.929276 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.929137 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:27.969176 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:27.969123 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" podStartSLOduration=1.5105551940000002 podStartE2EDuration="1.969105327s" totalImagesPullingTime="458.550133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:04:27.965895522 +0000 UTC m=+2343.774007541" watchObservedRunningTime="2026-07-16 17:04:27.969105327 +0000 UTC m=+2343.777217342" Jul 16 17:04:33.940239 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:33.940207 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:34.370358 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.370274 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6"] Jul 16 17:04:34.373804 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.373782 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.375997 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.375975 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6da193-serving-cert\"" Jul 16 17:04:34.383878 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.383853 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6"] Jul 16 17:04:34.409908 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.409871 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hppfr\" (UniqueName: \"kubernetes.io/projected/3fa62e50-0110-4167-a1e5-3ede59dac01f-kube-api-access-hppfr\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.410106 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.409933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa62e50-0110-4167-a1e5-3ede59dac01f-openshift-service-ca\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.410106 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.410008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fa62e50-0110-4167-a1e5-3ede59dac01f-maas-api-tls\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.511107 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.511036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa62e50-0110-4167-a1e5-3ede59dac01f-openshift-service-ca\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.511333 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.511156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fa62e50-0110-4167-a1e5-3ede59dac01f-maas-api-tls\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.511333 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.511199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hppfr\" (UniqueName: \"kubernetes.io/projected/3fa62e50-0110-4167-a1e5-3ede59dac01f-kube-api-access-hppfr\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.511707 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.511682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa62e50-0110-4167-a1e5-3ede59dac01f-openshift-service-ca\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.513699 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.513681 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fa62e50-0110-4167-a1e5-3ede59dac01f-maas-api-tls\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.532583 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.532554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hppfr\" (UniqueName: \"kubernetes.io/projected/3fa62e50-0110-4167-a1e5-3ede59dac01f-kube-api-access-hppfr\") pod \"maas-api-e2e-api-b-6da193-68d6849586-6zdt6\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.684912 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.684878 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:34.832265 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.832239 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6"] Jul 16 17:04:34.834263 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:04:34.834230 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa62e50_0110_4167_a1e5_3ede59dac01f.slice/crio-5bad03369fb4900a2e30c623c39817bf0e1a085ed8ab94cdcb7a0406f759e267 WatchSource:0}: Error finding container 5bad03369fb4900a2e30c623c39817bf0e1a085ed8ab94cdcb7a0406f759e267: Status 404 returned error can't find the container with id 5bad03369fb4900a2e30c623c39817bf0e1a085ed8ab94cdcb7a0406f759e267 Jul 16 17:04:34.968994 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:34.968901 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" event={"ID":"3fa62e50-0110-4167-a1e5-3ede59dac01f","Type":"ContainerStarted","Data":"5bad03369fb4900a2e30c623c39817bf0e1a085ed8ab94cdcb7a0406f759e267"} Jul 16 17:04:35.976470 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:35.976430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" event={"ID":"3fa62e50-0110-4167-a1e5-3ede59dac01f","Type":"ContainerStarted","Data":"fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716"} Jul 16 17:04:35.976890 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:35.976503 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:35.994398 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:35.994347 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" podStartSLOduration=1.666108594 podStartE2EDuration="1.994330758s" totalImagesPullingTime="328.222164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:04:35.992744991 +0000 UTC m=+2351.800857009" watchObservedRunningTime="2026-07-16 17:04:35.994330758 +0000 UTC m=+2351.802442772" Jul 16 17:04:41.988042 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:41.988014 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:04:57.959295 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:57.959252 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr"] Jul 16 17:04:57.959755 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:57.959532 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" podUID="f66eade8-0fea-4dde-9dac-bb338ff1d9a4" containerName="maas-api" containerID="cri-o://63139b2c88270a9fff497c90f851dd2e2446ef598e0e831dae6a1fcc19ed1426" gracePeriod=30 Jul 16 17:04:58.100392 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.100342 2560 generic.go:356] "Generic (PLEG): container finished" podID="f66eade8-0fea-4dde-9dac-bb338ff1d9a4" containerID="63139b2c88270a9fff497c90f851dd2e2446ef598e0e831dae6a1fcc19ed1426" exitCode=0 Jul 16 17:04:58.100553 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.100415 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" event={"ID":"f66eade8-0fea-4dde-9dac-bb338ff1d9a4","Type":"ContainerDied","Data":"63139b2c88270a9fff497c90f851dd2e2446ef598e0e831dae6a1fcc19ed1426"} Jul 16 17:04:58.205954 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.205928 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:58.309266 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.309160 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls\") pod \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " Jul 16 17:04:58.309266 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.309214 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-openshift-service-ca\") pod \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " Jul 16 17:04:58.309478 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.309276 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gxl4\" (UniqueName: \"kubernetes.io/projected/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-kube-api-access-6gxl4\") pod \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\" (UID: \"f66eade8-0fea-4dde-9dac-bb338ff1d9a4\") " Jul 16 17:04:58.309628 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.309603 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f66eade8-0fea-4dde-9dac-bb338ff1d9a4" (UID: "f66eade8-0fea-4dde-9dac-bb338ff1d9a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:04:58.311578 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.311548 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f66eade8-0fea-4dde-9dac-bb338ff1d9a4" (UID: "f66eade8-0fea-4dde-9dac-bb338ff1d9a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:04:58.311578 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.311572 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-kube-api-access-6gxl4" (OuterVolumeSpecName: "kube-api-access-6gxl4") pod "f66eade8-0fea-4dde-9dac-bb338ff1d9a4" (UID: "f66eade8-0fea-4dde-9dac-bb338ff1d9a4"). InnerVolumeSpecName "kube-api-access-6gxl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:04:58.410715 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.410667 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gxl4\" (UniqueName: \"kubernetes.io/projected/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-kube-api-access-6gxl4\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:04:58.410715 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.410708 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:04:58.410715 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:58.410721 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66eade8-0fea-4dde-9dac-bb338ff1d9a4-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:04:59.107596 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:59.107565 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" Jul 16 17:04:59.107596 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:59.107593 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr" event={"ID":"f66eade8-0fea-4dde-9dac-bb338ff1d9a4","Type":"ContainerDied","Data":"c47709d035103af985ab3da15ed2ac5df25c4e0fde4e0d4a8867d082521d790c"} Jul 16 17:04:59.108092 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:59.107651 2560 scope.go:117] "RemoveContainer" containerID="63139b2c88270a9fff497c90f851dd2e2446ef598e0e831dae6a1fcc19ed1426" Jul 16 17:04:59.124567 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:59.124537 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr"] Jul 16 17:04:59.128975 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:04:59.128950 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f111d-587994488b-jwkdr"] Jul 16 17:05:00.655519 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:00.655483 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f66eade8-0fea-4dde-9dac-bb338ff1d9a4" path="/var/lib/kubelet/pods/f66eade8-0fea-4dde-9dac-bb338ff1d9a4/volumes" Jul 16 17:05:23.828205 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:23.828050 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6"] Jul 16 17:05:23.828792 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:23.828442 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" podUID="3fa62e50-0110-4167-a1e5-3ede59dac01f" containerName="maas-api" containerID="cri-o://fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716" gracePeriod=30 Jul 16 17:05:24.081627 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.081561 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:05:24.238451 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.238411 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hppfr\" (UniqueName: \"kubernetes.io/projected/3fa62e50-0110-4167-a1e5-3ede59dac01f-kube-api-access-hppfr\") pod \"3fa62e50-0110-4167-a1e5-3ede59dac01f\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " Jul 16 17:05:24.238451 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.238450 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa62e50-0110-4167-a1e5-3ede59dac01f-openshift-service-ca\") pod \"3fa62e50-0110-4167-a1e5-3ede59dac01f\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " Jul 16 17:05:24.238664 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.238483 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fa62e50-0110-4167-a1e5-3ede59dac01f-maas-api-tls\") pod \"3fa62e50-0110-4167-a1e5-3ede59dac01f\" (UID: \"3fa62e50-0110-4167-a1e5-3ede59dac01f\") " Jul 16 17:05:24.238884 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.238856 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa62e50-0110-4167-a1e5-3ede59dac01f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fa62e50-0110-4167-a1e5-3ede59dac01f" (UID: "3fa62e50-0110-4167-a1e5-3ede59dac01f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:05:24.240798 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.240772 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fa62e50-0110-4167-a1e5-3ede59dac01f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fa62e50-0110-4167-a1e5-3ede59dac01f" (UID: "3fa62e50-0110-4167-a1e5-3ede59dac01f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:05:24.240946 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.240921 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa62e50-0110-4167-a1e5-3ede59dac01f-kube-api-access-hppfr" (OuterVolumeSpecName: "kube-api-access-hppfr") pod "3fa62e50-0110-4167-a1e5-3ede59dac01f" (UID: "3fa62e50-0110-4167-a1e5-3ede59dac01f"). InnerVolumeSpecName "kube-api-access-hppfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:05:24.248277 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.248249 2560 generic.go:356] "Generic (PLEG): container finished" podID="3fa62e50-0110-4167-a1e5-3ede59dac01f" containerID="fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716" exitCode=0 Jul 16 17:05:24.248428 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.248298 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" Jul 16 17:05:24.248428 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.248374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" event={"ID":"3fa62e50-0110-4167-a1e5-3ede59dac01f","Type":"ContainerDied","Data":"fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716"} Jul 16 17:05:24.248428 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.248424 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6" event={"ID":"3fa62e50-0110-4167-a1e5-3ede59dac01f","Type":"ContainerDied","Data":"5bad03369fb4900a2e30c623c39817bf0e1a085ed8ab94cdcb7a0406f759e267"} Jul 16 17:05:24.248584 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.248451 2560 scope.go:117] "RemoveContainer" containerID="fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716" Jul 16 17:05:24.258256 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.258235 2560 scope.go:117] "RemoveContainer" containerID="fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716" Jul 16 17:05:24.258543 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:05:24.258526 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716\": container with ID starting with fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716 not found: ID does not exist" containerID="fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716" Jul 16 17:05:24.258607 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.258552 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716"} err="failed to get container status \"fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716\": rpc error: code = NotFound desc = could not find container \"fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716\": container with ID starting with fb04cb086a0769bb9a7025d765501d57f4ee4e08035bfba684c4a7125108b716 not found: ID does not exist" Jul 16 17:05:24.270044 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.270015 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6"] Jul 16 17:05:24.272402 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.272381 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6da193-68d6849586-6zdt6"] Jul 16 17:05:24.339592 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.339499 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hppfr\" (UniqueName: \"kubernetes.io/projected/3fa62e50-0110-4167-a1e5-3ede59dac01f-kube-api-access-hppfr\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:05:24.339592 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.339530 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa62e50-0110-4167-a1e5-3ede59dac01f-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:05:24.339592 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.339542 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fa62e50-0110-4167-a1e5-3ede59dac01f-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:05:24.656263 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.656177 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fa62e50-0110-4167-a1e5-3ede59dac01f" path="/var/lib/kubelet/pods/3fa62e50-0110-4167-a1e5-3ede59dac01f/volumes" Jul 16 17:05:24.784196 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.784080 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:05:24.803051 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:24.803030 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:05:43.440682 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.440648 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz"] Jul 16 17:05:43.441193 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.441000 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66eade8-0fea-4dde-9dac-bb338ff1d9a4" containerName="maas-api" Jul 16 17:05:43.441193 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.441075 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66eade8-0fea-4dde-9dac-bb338ff1d9a4" containerName="maas-api" Jul 16 17:05:43.441193 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.441129 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa62e50-0110-4167-a1e5-3ede59dac01f" containerName="maas-api" Jul 16 17:05:43.441193 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.441139 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa62e50-0110-4167-a1e5-3ede59dac01f" containerName="maas-api" Jul 16 17:05:43.441375 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.441239 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66eade8-0fea-4dde-9dac-bb338ff1d9a4" containerName="maas-api" Jul 16 17:05:43.441375 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.441254 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa62e50-0110-4167-a1e5-3ede59dac01f" containerName="maas-api" Jul 16 17:05:43.444522 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.444506 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.446833 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.446803 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0cb2ac-serving-cert\"" Jul 16 17:05:43.446833 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.446826 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9c6vw\"" Jul 16 17:05:43.459365 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.459335 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz"] Jul 16 17:05:43.508916 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.508874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x26s\" (UniqueName: \"kubernetes.io/projected/778c6e91-9ad7-410d-9295-d7e6f9e0401d-kube-api-access-6x26s\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.509133 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.508955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778c6e91-9ad7-410d-9295-d7e6f9e0401d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.509133 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.509028 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778c6e91-9ad7-410d-9295-d7e6f9e0401d-maas-api-tls\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.610173 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.610127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x26s\" (UniqueName: \"kubernetes.io/projected/778c6e91-9ad7-410d-9295-d7e6f9e0401d-kube-api-access-6x26s\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.610384 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.610197 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778c6e91-9ad7-410d-9295-d7e6f9e0401d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.610384 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.610223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778c6e91-9ad7-410d-9295-d7e6f9e0401d-maas-api-tls\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.610871 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.610844 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778c6e91-9ad7-410d-9295-d7e6f9e0401d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.612846 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.612825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778c6e91-9ad7-410d-9295-d7e6f9e0401d-maas-api-tls\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.618264 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.618238 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x26s\" (UniqueName: \"kubernetes.io/projected/778c6e91-9ad7-410d-9295-d7e6f9e0401d-kube-api-access-6x26s\") pod \"maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.755885 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.755788 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:43.906997 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:43.906968 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz"] Jul 16 17:05:43.908849 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:05:43.908823 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778c6e91_9ad7_410d_9295_d7e6f9e0401d.slice/crio-ceb0832d0836787c92d056252a462433ccd9b530ec6076067915f4e60d90cf8d WatchSource:0}: Error finding container ceb0832d0836787c92d056252a462433ccd9b530ec6076067915f4e60d90cf8d: Status 404 returned error can't find the container with id ceb0832d0836787c92d056252a462433ccd9b530ec6076067915f4e60d90cf8d Jul 16 17:05:44.355114 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:44.355051 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" event={"ID":"778c6e91-9ad7-410d-9295-d7e6f9e0401d","Type":"ContainerStarted","Data":"ceb0832d0836787c92d056252a462433ccd9b530ec6076067915f4e60d90cf8d"} Jul 16 17:05:45.361357 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:45.361264 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" event={"ID":"778c6e91-9ad7-410d-9295-d7e6f9e0401d","Type":"ContainerStarted","Data":"1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa"} Jul 16 17:05:45.361357 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:45.361340 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:45.378550 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:45.378496 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" podStartSLOduration=1.9220332629999999 podStartE2EDuration="2.378479561s" totalImagesPullingTime="456.446298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:05:45.377439948 +0000 UTC m=+2421.185551961" watchObservedRunningTime="2026-07-16 17:05:45.378479561 +0000 UTC m=+2421.186591575" Jul 16 17:05:51.371163 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.371134 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:05:51.530858 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.530817 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf"] Jul 16 17:05:51.535145 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.535116 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.537278 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.537256 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-734b32-serving-cert\"" Jul 16 17:05:51.544706 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.544676 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf"] Jul 16 17:05:51.582641 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.582598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.582813 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.582660 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a95059c0-1512-42eb-90d4-c43e3d06ddb7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.582813 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.582750 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2r86\" (UniqueName: \"kubernetes.io/projected/a95059c0-1512-42eb-90d4-c43e3d06ddb7-kube-api-access-f2r86\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.683887 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.683851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.684086 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.683924 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a95059c0-1512-42eb-90d4-c43e3d06ddb7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.684086 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.683960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2r86\" (UniqueName: \"kubernetes.io/projected/a95059c0-1512-42eb-90d4-c43e3d06ddb7-kube-api-access-f2r86\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.684086 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:05:51.684016 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-serving-cert: secret "maas-api-e2e-shared-b-734b32-serving-cert" not found Jul 16 17:05:51.684246 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:05:51.684134 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls podName:a95059c0-1512-42eb-90d4-c43e3d06ddb7 nodeName:}" failed. No retries permitted until 2026-07-16 17:05:52.184110743 +0000 UTC m=+2427.992222748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls") pod "maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" (UID: "a95059c0-1512-42eb-90d4-c43e3d06ddb7") : secret "maas-api-e2e-shared-b-734b32-serving-cert" not found Jul 16 17:05:51.684598 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.684576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a95059c0-1512-42eb-90d4-c43e3d06ddb7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:51.699238 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:51.699204 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2r86\" (UniqueName: \"kubernetes.io/projected/a95059c0-1512-42eb-90d4-c43e3d06ddb7-kube-api-access-f2r86\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:52.187970 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:52.187930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:52.190635 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:52.190611 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls\") pod \"maas-api-e2e-shared-b-734b32-866bf4988-ttbmf\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:52.447807 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:52.447714 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:52.594335 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:52.594305 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf"] Jul 16 17:05:52.596136 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:05:52.596106 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95059c0_1512_42eb_90d4_c43e3d06ddb7.slice/crio-63bc427a1fad4837152c67c08232010498a4441323d928084b83561179161ecf WatchSource:0}: Error finding container 63bc427a1fad4837152c67c08232010498a4441323d928084b83561179161ecf: Status 404 returned error can't find the container with id 63bc427a1fad4837152c67c08232010498a4441323d928084b83561179161ecf Jul 16 17:05:53.405252 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:53.405206 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" event={"ID":"a95059c0-1512-42eb-90d4-c43e3d06ddb7","Type":"ContainerStarted","Data":"63bc427a1fad4837152c67c08232010498a4441323d928084b83561179161ecf"} Jul 16 17:05:54.412778 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:54.412739 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" event={"ID":"a95059c0-1512-42eb-90d4-c43e3d06ddb7","Type":"ContainerStarted","Data":"96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33"} Jul 16 17:05:54.413258 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:54.412900 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:05:54.435206 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:05:54.435154 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" podStartSLOduration=2.293406027 podStartE2EDuration="3.435136655s" totalImagesPullingTime="1.141730628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:05:54.433429362 +0000 UTC m=+2430.241541368" watchObservedRunningTime="2026-07-16 17:05:54.435136655 +0000 UTC m=+2430.243248668" Jul 16 17:06:00.423901 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:06:00.423872 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:08:24.944615 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:08:24.944583 2560 scope.go:117] "RemoveContainer" containerID="8dff2ec82cede251b78116ea12b19e948c265938ac2ba93843524918aa56e364" Jul 16 17:10:24.817307 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:10:24.817186 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:10:24.843196 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:10:24.843168 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:11:27.398942 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.398904 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7376-140485811469104"] Jul 16 17:11:27.402615 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.402591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:27.404992 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.404971 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4d6z2\"" Jul 16 17:11:27.411705 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.411677 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811469104"] Jul 16 17:11:27.437206 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.437175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47tkp\" (UniqueName: \"kubernetes.io/projected/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6-kube-api-access-47tkp\") pod \"test-curl-7376-140485811469104\" (UID: \"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6\") " pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:27.537988 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.537950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47tkp\" (UniqueName: \"kubernetes.io/projected/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6-kube-api-access-47tkp\") pod \"test-curl-7376-140485811469104\" (UID: \"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6\") " pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:27.549321 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.549293 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47tkp\" (UniqueName: \"kubernetes.io/projected/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6-kube-api-access-47tkp\") pod \"test-curl-7376-140485811469104\" (UID: \"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6\") " pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:27.713366 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.713335 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:27.857441 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.857414 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811469104"] Jul 16 17:11:27.858747 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:11:27.858721 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a234b16_dcb9_44c3_a5eb_4fa57a0437a6.slice/crio-83db135b677ad6d95377763ae28fc689afcc58a6d90c3bcb4826a6f3960fef4e WatchSource:0}: Error finding container 83db135b677ad6d95377763ae28fc689afcc58a6d90c3bcb4826a6f3960fef4e: Status 404 returned error can't find the container with id 83db135b677ad6d95377763ae28fc689afcc58a6d90c3bcb4826a6f3960fef4e Jul 16 17:11:27.860030 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:27.860014 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:11:28.214861 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:28.214791 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811469104" event={"ID":"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6","Type":"ContainerStarted","Data":"83db135b677ad6d95377763ae28fc689afcc58a6d90c3bcb4826a6f3960fef4e"} Jul 16 17:11:29.221238 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:29.221202 2560 generic.go:356] "Generic (PLEG): container finished" podID="4a234b16-dcb9-44c3-a5eb-4fa57a0437a6" containerID="e5a6687d9ab121562c808b30663b217e4e40e720d2367c16c1df3470bcefc046" exitCode=0 Jul 16 17:11:29.221644 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:29.221287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811469104" event={"ID":"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6","Type":"ContainerDied","Data":"e5a6687d9ab121562c808b30663b217e4e40e720d2367c16c1df3470bcefc046"} Jul 16 17:11:30.358383 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.358356 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:30.402911 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.402888 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7376-140485811469104_4a234b16-dcb9-44c3-a5eb-4fa57a0437a6/test-curl-7376-140485811469104/0.log" Jul 16 17:11:30.436198 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.436169 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7376-140485811469104"] Jul 16 17:11:30.439824 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.439800 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7376-140485811469104"] Jul 16 17:11:30.461946 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.461921 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47tkp\" (UniqueName: \"kubernetes.io/projected/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6-kube-api-access-47tkp\") pod \"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6\" (UID: \"4a234b16-dcb9-44c3-a5eb-4fa57a0437a6\") " Jul 16 17:11:30.464203 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.464176 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6-kube-api-access-47tkp" (OuterVolumeSpecName: "kube-api-access-47tkp") pod "4a234b16-dcb9-44c3-a5eb-4fa57a0437a6" (UID: "4a234b16-dcb9-44c3-a5eb-4fa57a0437a6"). InnerVolumeSpecName "kube-api-access-47tkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:11:30.563504 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.563418 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47tkp\" (UniqueName: \"kubernetes.io/projected/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6-kube-api-access-47tkp\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:30.565599 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.565578 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7376-140485811469232"] Jul 16 17:11:30.565927 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.565915 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a234b16-dcb9-44c3-a5eb-4fa57a0437a6" containerName="test-curl-7376-140485811469104" Jul 16 17:11:30.565965 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.565929 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a234b16-dcb9-44c3-a5eb-4fa57a0437a6" containerName="test-curl-7376-140485811469104" Jul 16 17:11:30.565999 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.565993 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a234b16-dcb9-44c3-a5eb-4fa57a0437a6" containerName="test-curl-7376-140485811469104" Jul 16 17:11:30.568965 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.568948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:30.575397 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.575372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811469232"] Jul 16 17:11:30.655470 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.655442 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a234b16-dcb9-44c3-a5eb-4fa57a0437a6" path="/var/lib/kubelet/pods/4a234b16-dcb9-44c3-a5eb-4fa57a0437a6/volumes" Jul 16 17:11:30.664617 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.664589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr6ff\" (UniqueName: \"kubernetes.io/projected/8f0d461d-1756-4cfb-83b4-f8425a288523-kube-api-access-zr6ff\") pod \"test-curl-7376-140485811469232\" (UID: \"8f0d461d-1756-4cfb-83b4-f8425a288523\") " pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:30.765976 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.765946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr6ff\" (UniqueName: \"kubernetes.io/projected/8f0d461d-1756-4cfb-83b4-f8425a288523-kube-api-access-zr6ff\") pod \"test-curl-7376-140485811469232\" (UID: \"8f0d461d-1756-4cfb-83b4-f8425a288523\") " pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:30.777639 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.777609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr6ff\" (UniqueName: \"kubernetes.io/projected/8f0d461d-1756-4cfb-83b4-f8425a288523-kube-api-access-zr6ff\") pod \"test-curl-7376-140485811469232\" (UID: \"8f0d461d-1756-4cfb-83b4-f8425a288523\") " pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:30.879393 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:30.879313 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:31.022105 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:31.022082 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811469232"] Jul 16 17:11:31.024212 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:11:31.024185 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f0d461d_1756_4cfb_83b4_f8425a288523.slice/crio-08b962c69e188838ded9fa574d43b5197522a6a3e330a1cc75230f85c5fe5725 WatchSource:0}: Error finding container 08b962c69e188838ded9fa574d43b5197522a6a3e330a1cc75230f85c5fe5725: Status 404 returned error can't find the container with id 08b962c69e188838ded9fa574d43b5197522a6a3e330a1cc75230f85c5fe5725 Jul 16 17:11:31.233334 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:31.233300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811469232" event={"ID":"8f0d461d-1756-4cfb-83b4-f8425a288523","Type":"ContainerStarted","Data":"08b962c69e188838ded9fa574d43b5197522a6a3e330a1cc75230f85c5fe5725"} Jul 16 17:11:31.234659 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:31.234642 2560 scope.go:117] "RemoveContainer" containerID="e5a6687d9ab121562c808b30663b217e4e40e720d2367c16c1df3470bcefc046" Jul 16 17:11:31.234659 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:31.234653 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469104" Jul 16 17:11:32.248958 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:32.248895 2560 generic.go:356] "Generic (PLEG): container finished" podID="8f0d461d-1756-4cfb-83b4-f8425a288523" containerID="4c04d60d6f279b3f2e9201eca339d139f87afca1adb1211d6fd4ae9921af07fe" exitCode=0 Jul 16 17:11:32.249464 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:32.248975 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811469232" event={"ID":"8f0d461d-1756-4cfb-83b4-f8425a288523","Type":"ContainerDied","Data":"4c04d60d6f279b3f2e9201eca339d139f87afca1adb1211d6fd4ae9921af07fe"} Jul 16 17:11:33.391756 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.391730 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:33.438610 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.438581 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7376-140485811469232_8f0d461d-1756-4cfb-83b4-f8425a288523/test-curl-7376-140485811469232/0.log" Jul 16 17:11:33.476118 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.476069 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7376-140485811469232"] Jul 16 17:11:33.484491 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.484466 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7376-140485811469232"] Jul 16 17:11:33.488903 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.488881 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr6ff\" (UniqueName: \"kubernetes.io/projected/8f0d461d-1756-4cfb-83b4-f8425a288523-kube-api-access-zr6ff\") pod \"8f0d461d-1756-4cfb-83b4-f8425a288523\" (UID: \"8f0d461d-1756-4cfb-83b4-f8425a288523\") " Jul 16 17:11:33.491037 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.491009 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0d461d-1756-4cfb-83b4-f8425a288523-kube-api-access-zr6ff" (OuterVolumeSpecName: "kube-api-access-zr6ff") pod "8f0d461d-1756-4cfb-83b4-f8425a288523" (UID: "8f0d461d-1756-4cfb-83b4-f8425a288523"). InnerVolumeSpecName "kube-api-access-zr6ff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:11:33.590289 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.590208 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr6ff\" (UniqueName: \"kubernetes.io/projected/8f0d461d-1756-4cfb-83b4-f8425a288523-kube-api-access-zr6ff\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:33.674958 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.674924 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:33.675306 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.675293 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0d461d-1756-4cfb-83b4-f8425a288523" containerName="test-curl-7376-140485811469232" Jul 16 17:11:33.675369 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.675307 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0d461d-1756-4cfb-83b4-f8425a288523" containerName="test-curl-7376-140485811469232" Jul 16 17:11:33.675405 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.675376 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0d461d-1756-4cfb-83b4-f8425a288523" containerName="test-curl-7376-140485811469232" Jul 16 17:11:33.678329 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.678310 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:33.685948 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.685926 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:33.792854 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.792810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm6qj\" (UniqueName: \"kubernetes.io/projected/34799302-923a-4ad6-a559-ef6f12b9367a-kube-api-access-fm6qj\") pod \"test-curl-7376-140485811458096\" (UID: \"34799302-923a-4ad6-a559-ef6f12b9367a\") " pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:33.893834 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.893751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm6qj\" (UniqueName: \"kubernetes.io/projected/34799302-923a-4ad6-a559-ef6f12b9367a-kube-api-access-fm6qj\") pod \"test-curl-7376-140485811458096\" (UID: \"34799302-923a-4ad6-a559-ef6f12b9367a\") " pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:33.902277 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.902254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm6qj\" (UniqueName: \"kubernetes.io/projected/34799302-923a-4ad6-a559-ef6f12b9367a-kube-api-access-fm6qj\") pod \"test-curl-7376-140485811458096\" (UID: \"34799302-923a-4ad6-a559-ef6f12b9367a\") " pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:33.987933 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:33.987899 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:34.131642 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:34.131617 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:34.133519 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:11:34.133485 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34799302_923a_4ad6_a559_ef6f12b9367a.slice/crio-ee3904c336abc813745ebf66dd9df4b6c228c876d1087934c8ce46a5990dd15e WatchSource:0}: Error finding container ee3904c336abc813745ebf66dd9df4b6c228c876d1087934c8ce46a5990dd15e: Status 404 returned error can't find the container with id ee3904c336abc813745ebf66dd9df4b6c228c876d1087934c8ce46a5990dd15e Jul 16 17:11:34.262235 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:34.262204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811458096" event={"ID":"34799302-923a-4ad6-a559-ef6f12b9367a","Type":"ContainerStarted","Data":"ee3904c336abc813745ebf66dd9df4b6c228c876d1087934c8ce46a5990dd15e"} Jul 16 17:11:34.263426 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:34.263406 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08b962c69e188838ded9fa574d43b5197522a6a3e330a1cc75230f85c5fe5725" Jul 16 17:11:34.263547 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:34.263437 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:34.273034 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:11:34.273008 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7376-140485811469232\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" podUID="8f0d461d-1756-4cfb-83b4-f8425a288523" pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:34.654811 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:11:34.654768 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7376-140485811469232\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" podUID="8f0d461d-1756-4cfb-83b4-f8425a288523" pod="opendatahub/test-curl-7376-140485811469232" Jul 16 17:11:34.657085 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:34.657038 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0d461d-1756-4cfb-83b4-f8425a288523" path="/var/lib/kubelet/pods/8f0d461d-1756-4cfb-83b4-f8425a288523/volumes" Jul 16 17:11:35.270323 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:35.270286 2560 generic.go:356] "Generic (PLEG): container finished" podID="34799302-923a-4ad6-a559-ef6f12b9367a" containerID="9218f523f7121a0945c269084b2563756cf43df24cf6bf46833183edc3b21ab1" exitCode=0 Jul 16 17:11:35.270495 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:35.270379 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811458096" event={"ID":"34799302-923a-4ad6-a559-ef6f12b9367a","Type":"ContainerDied","Data":"9218f523f7121a0945c269084b2563756cf43df24cf6bf46833183edc3b21ab1"} Jul 16 17:11:36.416275 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.416244 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:36.444229 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.444207 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7376-140485811458096_34799302-923a-4ad6-a559-ef6f12b9367a/test-curl-7376-140485811458096/0.log" Jul 16 17:11:36.478421 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.478389 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:36.485174 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.485154 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:36.516315 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.516294 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm6qj\" (UniqueName: \"kubernetes.io/projected/34799302-923a-4ad6-a559-ef6f12b9367a-kube-api-access-fm6qj\") pod \"34799302-923a-4ad6-a559-ef6f12b9367a\" (UID: \"34799302-923a-4ad6-a559-ef6f12b9367a\") " Jul 16 17:11:36.518485 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.518456 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34799302-923a-4ad6-a559-ef6f12b9367a-kube-api-access-fm6qj" (OuterVolumeSpecName: "kube-api-access-fm6qj") pod "34799302-923a-4ad6-a559-ef6f12b9367a" (UID: "34799302-923a-4ad6-a559-ef6f12b9367a"). InnerVolumeSpecName "kube-api-access-fm6qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:11:36.617720 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.617642 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm6qj\" (UniqueName: \"kubernetes.io/projected/34799302-923a-4ad6-a559-ef6f12b9367a-kube-api-access-fm6qj\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:36.618746 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.618722 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:36.619136 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.619123 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34799302-923a-4ad6-a559-ef6f12b9367a" containerName="test-curl-7376-140485811458096" Jul 16 17:11:36.619222 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.619138 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="34799302-923a-4ad6-a559-ef6f12b9367a" containerName="test-curl-7376-140485811458096" Jul 16 17:11:36.619222 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.619203 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="34799302-923a-4ad6-a559-ef6f12b9367a" containerName="test-curl-7376-140485811458096" Jul 16 17:11:36.622208 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.622192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:36.632751 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.632728 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:36.655874 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.655849 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34799302-923a-4ad6-a559-ef6f12b9367a" path="/var/lib/kubelet/pods/34799302-923a-4ad6-a559-ef6f12b9367a/volumes" Jul 16 17:11:36.719003 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.718972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6vd9\" (UniqueName: \"kubernetes.io/projected/5766f47e-d4e9-464a-a48b-40c1c9696ce0-kube-api-access-q6vd9\") pod \"test-curl-7376-140485811458096\" (UID: \"5766f47e-d4e9-464a-a48b-40c1c9696ce0\") " pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:36.820481 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.820448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6vd9\" (UniqueName: \"kubernetes.io/projected/5766f47e-d4e9-464a-a48b-40c1c9696ce0-kube-api-access-q6vd9\") pod \"test-curl-7376-140485811458096\" (UID: \"5766f47e-d4e9-464a-a48b-40c1c9696ce0\") " pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:36.830679 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.830650 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6vd9\" (UniqueName: \"kubernetes.io/projected/5766f47e-d4e9-464a-a48b-40c1c9696ce0-kube-api-access-q6vd9\") pod \"test-curl-7376-140485811458096\" (UID: \"5766f47e-d4e9-464a-a48b-40c1c9696ce0\") " pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:36.931535 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:36.931501 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:37.071436 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:37.071411 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:37.073808 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:11:37.073777 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5766f47e_d4e9_464a_a48b_40c1c9696ce0.slice/crio-f311366a370cff0e70dc8c1eb529c306ab72b0e8d8a9801a36162451b80b16e0 WatchSource:0}: Error finding container f311366a370cff0e70dc8c1eb529c306ab72b0e8d8a9801a36162451b80b16e0: Status 404 returned error can't find the container with id f311366a370cff0e70dc8c1eb529c306ab72b0e8d8a9801a36162451b80b16e0 Jul 16 17:11:37.282980 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:37.282899 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:37.282980 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:37.282926 2560 scope.go:117] "RemoveContainer" containerID="9218f523f7121a0945c269084b2563756cf43df24cf6bf46833183edc3b21ab1" Jul 16 17:11:37.284231 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:37.284205 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811458096" event={"ID":"5766f47e-d4e9-464a-a48b-40c1c9696ce0","Type":"ContainerStarted","Data":"f311366a370cff0e70dc8c1eb529c306ab72b0e8d8a9801a36162451b80b16e0"} Jul 16 17:11:38.291376 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:38.291332 2560 generic.go:356] "Generic (PLEG): container finished" podID="5766f47e-d4e9-464a-a48b-40c1c9696ce0" containerID="f13a9fc19fcebfdc5a79e1f3b4f10157cf4ecfae7b0c498d357f28ee53aeb1f6" exitCode=0 Jul 16 17:11:38.291773 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:38.291424 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811458096" event={"ID":"5766f47e-d4e9-464a-a48b-40c1c9696ce0","Type":"ContainerDied","Data":"f13a9fc19fcebfdc5a79e1f3b4f10157cf4ecfae7b0c498d357f28ee53aeb1f6"} Jul 16 17:11:39.436145 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.436119 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:39.486033 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.486010 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7376-140485811458096_5766f47e-d4e9-464a-a48b-40c1c9696ce0/test-curl-7376-140485811458096/0.log" Jul 16 17:11:39.530138 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.530107 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:39.534566 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.534544 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7376-140485811458096"] Jul 16 17:11:39.545505 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.545479 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6vd9\" (UniqueName: \"kubernetes.io/projected/5766f47e-d4e9-464a-a48b-40c1c9696ce0-kube-api-access-q6vd9\") pod \"5766f47e-d4e9-464a-a48b-40c1c9696ce0\" (UID: \"5766f47e-d4e9-464a-a48b-40c1c9696ce0\") " Jul 16 17:11:39.547731 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.547673 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5766f47e-d4e9-464a-a48b-40c1c9696ce0-kube-api-access-q6vd9" (OuterVolumeSpecName: "kube-api-access-q6vd9") pod "5766f47e-d4e9-464a-a48b-40c1c9696ce0" (UID: "5766f47e-d4e9-464a-a48b-40c1c9696ce0"). InnerVolumeSpecName "kube-api-access-q6vd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:11:39.646313 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.646279 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6vd9\" (UniqueName: \"kubernetes.io/projected/5766f47e-d4e9-464a-a48b-40c1c9696ce0-kube-api-access-q6vd9\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:39.661667 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.661637 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7376-140485811459104"] Jul 16 17:11:39.662019 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.662006 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5766f47e-d4e9-464a-a48b-40c1c9696ce0" containerName="test-curl-7376-140485811458096" Jul 16 17:11:39.662073 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.662021 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5766f47e-d4e9-464a-a48b-40c1c9696ce0" containerName="test-curl-7376-140485811458096" Jul 16 17:11:39.662126 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.662093 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5766f47e-d4e9-464a-a48b-40c1c9696ce0" containerName="test-curl-7376-140485811458096" Jul 16 17:11:39.664964 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.664949 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:39.671929 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.671772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811459104"] Jul 16 17:11:39.747461 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.747433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbpfb\" (UniqueName: \"kubernetes.io/projected/ab65fe4e-8daf-474a-80c9-a9386ff970fd-kube-api-access-zbpfb\") pod \"test-curl-7376-140485811459104\" (UID: \"ab65fe4e-8daf-474a-80c9-a9386ff970fd\") " pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:39.848431 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.848353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbpfb\" (UniqueName: \"kubernetes.io/projected/ab65fe4e-8daf-474a-80c9-a9386ff970fd-kube-api-access-zbpfb\") pod \"test-curl-7376-140485811459104\" (UID: \"ab65fe4e-8daf-474a-80c9-a9386ff970fd\") " pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:39.857794 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.857769 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbpfb\" (UniqueName: \"kubernetes.io/projected/ab65fe4e-8daf-474a-80c9-a9386ff970fd-kube-api-access-zbpfb\") pod \"test-curl-7376-140485811459104\" (UID: \"ab65fe4e-8daf-474a-80c9-a9386ff970fd\") " pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:39.974791 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:39.974741 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:40.114417 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:40.114391 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7376-140485811459104"] Jul 16 17:11:40.116384 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:11:40.116354 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab65fe4e_8daf_474a_80c9_a9386ff970fd.slice/crio-81fc8f890a48eee9c667e48409c6684b5542820773c69f5962cd4542d15715eb WatchSource:0}: Error finding container 81fc8f890a48eee9c667e48409c6684b5542820773c69f5962cd4542d15715eb: Status 404 returned error can't find the container with id 81fc8f890a48eee9c667e48409c6684b5542820773c69f5962cd4542d15715eb Jul 16 17:11:40.303378 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:40.303350 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f311366a370cff0e70dc8c1eb529c306ab72b0e8d8a9801a36162451b80b16e0" Jul 16 17:11:40.303378 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:40.303361 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:40.304521 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:40.304495 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811459104" event={"ID":"ab65fe4e-8daf-474a-80c9-a9386ff970fd","Type":"ContainerStarted","Data":"81fc8f890a48eee9c667e48409c6684b5542820773c69f5962cd4542d15715eb"} Jul 16 17:11:40.314447 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:11:40.314421 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7376-140485811458096\" is forbidden: User \"system:node:ip-10-0-131-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-94.ec2.internal' and this object" podUID="5766f47e-d4e9-464a-a48b-40c1c9696ce0" pod="opendatahub/test-curl-7376-140485811458096" Jul 16 17:11:40.656114 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:40.656071 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5766f47e-d4e9-464a-a48b-40c1c9696ce0" path="/var/lib/kubelet/pods/5766f47e-d4e9-464a-a48b-40c1c9696ce0/volumes" Jul 16 17:11:41.311169 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:41.311132 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab65fe4e-8daf-474a-80c9-a9386ff970fd" containerID="2ac92e3ea73faf16694666f4a0610a5cf4cb61234eb3b4608ad9a1d301a0bd26" exitCode=0 Jul 16 17:11:41.311347 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:41.311229 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7376-140485811459104" event={"ID":"ab65fe4e-8daf-474a-80c9-a9386ff970fd","Type":"ContainerDied","Data":"2ac92e3ea73faf16694666f4a0610a5cf4cb61234eb3b4608ad9a1d301a0bd26"} Jul 16 17:11:42.470712 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.470688 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:42.499450 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.499413 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7376-140485811459104_ab65fe4e-8daf-474a-80c9-a9386ff970fd/test-curl-7376-140485811459104/0.log" Jul 16 17:11:42.531772 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.531742 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7376-140485811459104"] Jul 16 17:11:42.535808 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.535780 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7376-140485811459104"] Jul 16 17:11:42.574868 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.574785 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbpfb\" (UniqueName: \"kubernetes.io/projected/ab65fe4e-8daf-474a-80c9-a9386ff970fd-kube-api-access-zbpfb\") pod \"ab65fe4e-8daf-474a-80c9-a9386ff970fd\" (UID: \"ab65fe4e-8daf-474a-80c9-a9386ff970fd\") " Jul 16 17:11:42.577077 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.577036 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab65fe4e-8daf-474a-80c9-a9386ff970fd-kube-api-access-zbpfb" (OuterVolumeSpecName: "kube-api-access-zbpfb") pod "ab65fe4e-8daf-474a-80c9-a9386ff970fd" (UID: "ab65fe4e-8daf-474a-80c9-a9386ff970fd"). InnerVolumeSpecName "kube-api-access-zbpfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:11:42.655854 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.655816 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab65fe4e-8daf-474a-80c9-a9386ff970fd" path="/var/lib/kubelet/pods/ab65fe4e-8daf-474a-80c9-a9386ff970fd/volumes" Jul 16 17:11:42.676276 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:42.676243 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbpfb\" (UniqueName: \"kubernetes.io/projected/ab65fe4e-8daf-474a-80c9-a9386ff970fd-kube-api-access-zbpfb\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:43.323231 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:43.323200 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7376-140485811459104" Jul 16 17:11:43.323393 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:43.323209 2560 scope.go:117] "RemoveContainer" containerID="2ac92e3ea73faf16694666f4a0610a5cf4cb61234eb3b4608ad9a1d301a0bd26" Jul 16 17:11:56.559973 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.559932 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz"] Jul 16 17:11:56.560410 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.560161 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" podUID="778c6e91-9ad7-410d-9295-d7e6f9e0401d" containerName="maas-api" containerID="cri-o://1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa" gracePeriod=30 Jul 16 17:11:56.825717 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.825650 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:11:56.904928 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.904893 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778c6e91-9ad7-410d-9295-d7e6f9e0401d-openshift-service-ca\") pod \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " Jul 16 17:11:56.904928 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.904943 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x26s\" (UniqueName: \"kubernetes.io/projected/778c6e91-9ad7-410d-9295-d7e6f9e0401d-kube-api-access-6x26s\") pod \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " Jul 16 17:11:56.905217 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.904982 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778c6e91-9ad7-410d-9295-d7e6f9e0401d-maas-api-tls\") pod \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\" (UID: \"778c6e91-9ad7-410d-9295-d7e6f9e0401d\") " Jul 16 17:11:56.905326 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.905301 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/778c6e91-9ad7-410d-9295-d7e6f9e0401d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "778c6e91-9ad7-410d-9295-d7e6f9e0401d" (UID: "778c6e91-9ad7-410d-9295-d7e6f9e0401d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:11:56.907245 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.907219 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778c6e91-9ad7-410d-9295-d7e6f9e0401d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "778c6e91-9ad7-410d-9295-d7e6f9e0401d" (UID: "778c6e91-9ad7-410d-9295-d7e6f9e0401d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:11:56.907318 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:56.907242 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778c6e91-9ad7-410d-9295-d7e6f9e0401d-kube-api-access-6x26s" (OuterVolumeSpecName: "kube-api-access-6x26s") pod "778c6e91-9ad7-410d-9295-d7e6f9e0401d" (UID: "778c6e91-9ad7-410d-9295-d7e6f9e0401d"). InnerVolumeSpecName "kube-api-access-6x26s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:11:57.006281 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.006249 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778c6e91-9ad7-410d-9295-d7e6f9e0401d-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:57.006281 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.006276 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x26s\" (UniqueName: \"kubernetes.io/projected/778c6e91-9ad7-410d-9295-d7e6f9e0401d-kube-api-access-6x26s\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:57.006281 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.006285 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778c6e91-9ad7-410d-9295-d7e6f9e0401d-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:11:57.399252 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.399211 2560 generic.go:356] "Generic (PLEG): container finished" podID="778c6e91-9ad7-410d-9295-d7e6f9e0401d" containerID="1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa" exitCode=0 Jul 16 17:11:57.399480 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.399260 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" Jul 16 17:11:57.399480 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.399263 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" event={"ID":"778c6e91-9ad7-410d-9295-d7e6f9e0401d","Type":"ContainerDied","Data":"1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa"} Jul 16 17:11:57.399480 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.399355 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz" event={"ID":"778c6e91-9ad7-410d-9295-d7e6f9e0401d","Type":"ContainerDied","Data":"ceb0832d0836787c92d056252a462433ccd9b530ec6076067915f4e60d90cf8d"} Jul 16 17:11:57.399480 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.399388 2560 scope.go:117] "RemoveContainer" containerID="1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa" Jul 16 17:11:57.409859 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.409839 2560 scope.go:117] "RemoveContainer" containerID="1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa" Jul 16 17:11:57.410176 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:11:57.410155 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa\": container with ID starting with 1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa not found: ID does not exist" containerID="1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa" Jul 16 17:11:57.410257 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.410185 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa"} err="failed to get container status \"1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa\": rpc error: code = NotFound desc = could not find container \"1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa\": container with ID starting with 1766abf1dae7295e85de52dcfbe278247dc7fe8725894e0261ea187e2f6d25aa not found: ID does not exist" Jul 16 17:11:57.421347 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.421322 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz"] Jul 16 17:11:57.426440 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:57.426414 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0cb2ac-5bbb64bc47-9ptwz"] Jul 16 17:11:58.655807 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:11:58.655771 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="778c6e91-9ad7-410d-9295-d7e6f9e0401d" path="/var/lib/kubelet/pods/778c6e91-9ad7-410d-9295-d7e6f9e0401d/volumes" Jul 16 17:12:22.647961 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.647923 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf"] Jul 16 17:12:22.648520 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.648149 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" podUID="a95059c0-1512-42eb-90d4-c43e3d06ddb7" containerName="maas-api" containerID="cri-o://96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33" gracePeriod=30 Jul 16 17:12:22.901346 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.901275 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:12:22.925583 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.925547 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a95059c0-1512-42eb-90d4-c43e3d06ddb7-openshift-service-ca\") pod \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " Jul 16 17:12:22.925583 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.925583 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls\") pod \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " Jul 16 17:12:22.925834 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.925603 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2r86\" (UniqueName: \"kubernetes.io/projected/a95059c0-1512-42eb-90d4-c43e3d06ddb7-kube-api-access-f2r86\") pod \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\" (UID: \"a95059c0-1512-42eb-90d4-c43e3d06ddb7\") " Jul 16 17:12:22.925931 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.925904 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a95059c0-1512-42eb-90d4-c43e3d06ddb7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a95059c0-1512-42eb-90d4-c43e3d06ddb7" (UID: "a95059c0-1512-42eb-90d4-c43e3d06ddb7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:12:22.927881 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.927852 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a95059c0-1512-42eb-90d4-c43e3d06ddb7" (UID: "a95059c0-1512-42eb-90d4-c43e3d06ddb7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:12:22.927990 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:22.927901 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a95059c0-1512-42eb-90d4-c43e3d06ddb7-kube-api-access-f2r86" (OuterVolumeSpecName: "kube-api-access-f2r86") pod "a95059c0-1512-42eb-90d4-c43e3d06ddb7" (UID: "a95059c0-1512-42eb-90d4-c43e3d06ddb7"). InnerVolumeSpecName "kube-api-access-f2r86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:12:23.026466 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.026413 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a95059c0-1512-42eb-90d4-c43e3d06ddb7-openshift-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:12:23.026466 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.026459 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a95059c0-1512-42eb-90d4-c43e3d06ddb7-maas-api-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:12:23.026466 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.026469 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2r86\" (UniqueName: \"kubernetes.io/projected/a95059c0-1512-42eb-90d4-c43e3d06ddb7-kube-api-access-f2r86\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Jul 16 17:12:23.545179 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.545139 2560 generic.go:356] "Generic (PLEG): container finished" podID="a95059c0-1512-42eb-90d4-c43e3d06ddb7" containerID="96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33" exitCode=0 Jul 16 17:12:23.545179 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.545181 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" Jul 16 17:12:23.545450 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.545228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" event={"ID":"a95059c0-1512-42eb-90d4-c43e3d06ddb7","Type":"ContainerDied","Data":"96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33"} Jul 16 17:12:23.545450 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.545286 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf" event={"ID":"a95059c0-1512-42eb-90d4-c43e3d06ddb7","Type":"ContainerDied","Data":"63bc427a1fad4837152c67c08232010498a4441323d928084b83561179161ecf"} Jul 16 17:12:23.545450 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.545318 2560 scope.go:117] "RemoveContainer" containerID="96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33" Jul 16 17:12:23.558346 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.558326 2560 scope.go:117] "RemoveContainer" containerID="96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33" Jul 16 17:12:23.558653 ip-10-0-131-94 kubenswrapper[2560]: E0716 17:12:23.558631 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33\": container with ID starting with 96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33 not found: ID does not exist" containerID="96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33" Jul 16 17:12:23.558738 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.558660 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33"} err="failed to get container status \"96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33\": rpc error: code = NotFound desc = could not find container \"96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33\": container with ID starting with 96c1606bc0f957675f5417ca85235c232678441dd18e14669bdd207e3ab3dc33 not found: ID does not exist" Jul 16 17:12:23.568540 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.568510 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf"] Jul 16 17:12:23.575530 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:23.575508 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-734b32-866bf4988-ttbmf"] Jul 16 17:12:24.655997 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:24.655967 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a95059c0-1512-42eb-90d4-c43e3d06ddb7" path="/var/lib/kubelet/pods/a95059c0-1512-42eb-90d4-c43e3d06ddb7/volumes" Jul 16 17:12:38.550620 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:38.550566 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56df85b9f9-zmsvx_0a67dd93-cab8-425a-9ab5-bfe043ca1379/authorino/0.log" Jul 16 17:12:44.532631 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:44.532594 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6847bf9b68-kxdlt_bb35eee2-eba0-42bb-97e7-90dfd9b170ac/manager/0.log" Jul 16 17:12:44.779746 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:44.779703 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dfhzq_7a768f94-dae6-4570-99ea-87b24a7c3dd4/manager/0.log" Jul 16 17:12:44.910590 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:44.910505 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846bd4c774-bgllj_b25dc11b-50e7-49ec-8183-0bf7918b9ffb/copy-manifests/0.log" Jul 16 17:12:44.928556 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:44.928498 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846bd4c774-bgllj_b25dc11b-50e7-49ec-8183-0bf7918b9ffb/manager/0.log" Jul 16 17:12:45.597186 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:45.597159 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7664fcc798-bfw2m_1fd2a36f-ecf9-42c9-a272-d209a6a27042/maas-api/0.log" Jul 16 17:12:48.267359 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.267331 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q_b617f699-e76c-4314-abc3-fecd3bc20468/pull/0.log" Jul 16 17:12:48.276051 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.276021 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q_b617f699-e76c-4314-abc3-fecd3bc20468/extract/0.log" Jul 16 17:12:48.287288 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.287263 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q_b617f699-e76c-4314-abc3-fecd3bc20468/util/0.log" Jul 16 17:12:48.404471 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.404437 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z_92f49632-cded-4ff8-b203-f9b404405bac/pull/0.log" Jul 16 17:12:48.412953 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.412928 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z_92f49632-cded-4ff8-b203-f9b404405bac/extract/0.log" Jul 16 17:12:48.420492 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.420465 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z_92f49632-cded-4ff8-b203-f9b404405bac/util/0.log" Jul 16 17:12:48.537639 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.537544 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h_d9784c48-00dc-4a0e-a9da-43cb4886c638/util/0.log" Jul 16 17:12:48.548090 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.548047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h_d9784c48-00dc-4a0e-a9da-43cb4886c638/pull/0.log" Jul 16 17:12:48.555684 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.555662 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h_d9784c48-00dc-4a0e-a9da-43cb4886c638/extract/0.log" Jul 16 17:12:48.673200 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.673159 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z_7e8fe052-d9c9-4c37-b6b7-f5f2252d6673/util/0.log" Jul 16 17:12:48.681940 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.681899 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z_7e8fe052-d9c9-4c37-b6b7-f5f2252d6673/pull/0.log" Jul 16 17:12:48.690118 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.690083 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z_7e8fe052-d9c9-4c37-b6b7-f5f2252d6673/extract/0.log" Jul 16 17:12:48.822993 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:48.822890 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56df85b9f9-zmsvx_0a67dd93-cab8-425a-9ab5-bfe043ca1379/authorino/0.log" Jul 16 17:12:50.367892 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:50.367859 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6747f69598-6gkbs_c8930eed-7dda-4ce3-a11f-3e9cc46f62ce/kube-auth-proxy/0.log" Jul 16 17:12:50.985543 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:50.985515 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fb97c79fd-4cvvx_2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87/router/0.log" Jul 16 17:12:51.451487 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:51.451462 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-xlgsp_b36e52a4-3ae2-4b5f-b688-867bae24fb61/main/0.log" Jul 16 17:12:51.684107 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:51.684045 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbjpqzw_84431927-83f7-414f-84d0-beceffb4b972/main/0.log" Jul 16 17:12:51.921044 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:51.921015 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pjhmp_ec147ede-4d15-430c-9c29-89d221e1cdb2/main/0.log" Jul 16 17:12:57.602430 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602345 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-84tj4/must-gather-8nsvq"] Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602696 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a95059c0-1512-42eb-90d4-c43e3d06ddb7" containerName="maas-api" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602707 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a95059c0-1512-42eb-90d4-c43e3d06ddb7" containerName="maas-api" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602726 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778c6e91-9ad7-410d-9295-d7e6f9e0401d" containerName="maas-api" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602731 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="778c6e91-9ad7-410d-9295-d7e6f9e0401d" containerName="maas-api" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602737 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab65fe4e-8daf-474a-80c9-a9386ff970fd" containerName="test-curl-7376-140485811459104" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602744 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab65fe4e-8daf-474a-80c9-a9386ff970fd" containerName="test-curl-7376-140485811459104" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602796 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="778c6e91-9ad7-410d-9295-d7e6f9e0401d" containerName="maas-api" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602805 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a95059c0-1512-42eb-90d4-c43e3d06ddb7" containerName="maas-api" Jul 16 17:12:57.602838 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.602813 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab65fe4e-8daf-474a-80c9-a9386ff970fd" containerName="test-curl-7376-140485811459104" Jul 16 17:12:57.605783 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.605765 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.608611 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.608578 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-84tj4\"/\"openshift-service-ca.crt\"" Jul 16 17:12:57.608777 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.608580 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-84tj4\"/\"kube-root-ca.crt\"" Jul 16 17:12:57.608777 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.608718 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-84tj4\"/\"default-dockercfg-c7czs\"" Jul 16 17:12:57.621192 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.621164 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84tj4/must-gather-8nsvq"] Jul 16 17:12:57.743921 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.743867 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pznsj\" (UniqueName: \"kubernetes.io/projected/f961c5e2-3c87-40f3-89f7-500328c7973a-kube-api-access-pznsj\") pod \"must-gather-8nsvq\" (UID: \"f961c5e2-3c87-40f3-89f7-500328c7973a\") " pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.744186 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.743941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f961c5e2-3c87-40f3-89f7-500328c7973a-must-gather-output\") pod \"must-gather-8nsvq\" (UID: \"f961c5e2-3c87-40f3-89f7-500328c7973a\") " pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.845433 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.845389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pznsj\" (UniqueName: \"kubernetes.io/projected/f961c5e2-3c87-40f3-89f7-500328c7973a-kube-api-access-pznsj\") pod \"must-gather-8nsvq\" (UID: \"f961c5e2-3c87-40f3-89f7-500328c7973a\") " pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.845657 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.845440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f961c5e2-3c87-40f3-89f7-500328c7973a-must-gather-output\") pod \"must-gather-8nsvq\" (UID: \"f961c5e2-3c87-40f3-89f7-500328c7973a\") " pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.845759 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.845740 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f961c5e2-3c87-40f3-89f7-500328c7973a-must-gather-output\") pod \"must-gather-8nsvq\" (UID: \"f961c5e2-3c87-40f3-89f7-500328c7973a\") " pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.853667 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.853594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pznsj\" (UniqueName: \"kubernetes.io/projected/f961c5e2-3c87-40f3-89f7-500328c7973a-kube-api-access-pznsj\") pod \"must-gather-8nsvq\" (UID: \"f961c5e2-3c87-40f3-89f7-500328c7973a\") " pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:57.915684 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:57.915646 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84tj4/must-gather-8nsvq" Jul 16 17:12:58.056360 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:58.056328 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84tj4/must-gather-8nsvq"] Jul 16 17:12:58.058219 ip-10-0-131-94 kubenswrapper[2560]: W0716 17:12:58.058191 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf961c5e2_3c87_40f3_89f7_500328c7973a.slice/crio-a1821aa99ab23ca6c7558cfb978c7a043e6fd269a894ece82ac2a8b63bb79aff WatchSource:0}: Error finding container a1821aa99ab23ca6c7558cfb978c7a043e6fd269a894ece82ac2a8b63bb79aff: Status 404 returned error can't find the container with id a1821aa99ab23ca6c7558cfb978c7a043e6fd269a894ece82ac2a8b63bb79aff Jul 16 17:12:58.732372 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:58.732337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84tj4/must-gather-8nsvq" event={"ID":"f961c5e2-3c87-40f3-89f7-500328c7973a","Type":"ContainerStarted","Data":"a1821aa99ab23ca6c7558cfb978c7a043e6fd269a894ece82ac2a8b63bb79aff"} Jul 16 17:12:59.744885 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:59.744849 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84tj4/must-gather-8nsvq" event={"ID":"f961c5e2-3c87-40f3-89f7-500328c7973a","Type":"ContainerStarted","Data":"2ff6cdd466628a54e5426fe23a236600fd1879b2dd7cfe714f2875852503a316"} Jul 16 17:12:59.745381 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:59.744898 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84tj4/must-gather-8nsvq" event={"ID":"f961c5e2-3c87-40f3-89f7-500328c7973a","Type":"ContainerStarted","Data":"7853c10492323b07a529a053afd32d8628c15b17d33da9a25ec66902f4813b93"} Jul 16 17:12:59.780765 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:12:59.780710 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-84tj4/must-gather-8nsvq" podStartSLOduration=1.925364637 podStartE2EDuration="2.780692961s" totalImagesPullingTime="855.328324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:12:59.778955273 +0000 UTC m=+2855.587067287" watchObservedRunningTime="2026-07-16 17:12:59.780692961 +0000 UTC m=+2855.588804975" Jul 16 17:13:00.601730 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:00.601701 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nsqs2_1ffb1bb7-d510-455e-828d-88887fa3c651/global-pull-secret-syncer/0.log" Jul 16 17:13:00.784011 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:00.783973 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vfkkz_01259a01-eb1b-4e3f-b57a-ad02e9aad59d/konnectivity-agent/0.log" Jul 16 17:13:00.880762 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:00.880682 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-94.ec2.internal_3c01f07008c2f7f9b6faadbc640e87b1/haproxy/0.log" Jul 16 17:13:04.982100 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:04.982011 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q_b617f699-e76c-4314-abc3-fecd3bc20468/extract/0.log" Jul 16 17:13:05.011393 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.011338 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q_b617f699-e76c-4314-abc3-fecd3bc20468/util/0.log" Jul 16 17:13:05.037714 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.037632 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jg79q_b617f699-e76c-4314-abc3-fecd3bc20468/pull/0.log" Jul 16 17:13:05.081528 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.081488 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z_92f49632-cded-4ff8-b203-f9b404405bac/extract/0.log" Jul 16 17:13:05.108458 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.108420 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z_92f49632-cded-4ff8-b203-f9b404405bac/util/0.log" Jul 16 17:13:05.137965 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.137917 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09wx6z_92f49632-cded-4ff8-b203-f9b404405bac/pull/0.log" Jul 16 17:13:05.182072 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.181988 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h_d9784c48-00dc-4a0e-a9da-43cb4886c638/extract/0.log" Jul 16 17:13:05.218365 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.218332 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h_d9784c48-00dc-4a0e-a9da-43cb4886c638/util/0.log" Jul 16 17:13:05.251933 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.251823 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pgw7h_d9784c48-00dc-4a0e-a9da-43cb4886c638/pull/0.log" Jul 16 17:13:05.295504 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.295475 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z_7e8fe052-d9c9-4c37-b6b7-f5f2252d6673/extract/0.log" Jul 16 17:13:05.322265 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.322231 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z_7e8fe052-d9c9-4c37-b6b7-f5f2252d6673/util/0.log" Jul 16 17:13:05.349556 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.349483 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brk4z_7e8fe052-d9c9-4c37-b6b7-f5f2252d6673/pull/0.log" Jul 16 17:13:05.784606 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:05.784560 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56df85b9f9-zmsvx_0a67dd93-cab8-425a-9ab5-bfe043ca1379/authorino/0.log" Jul 16 17:13:08.065617 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:08.065479 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-66skd_2d1f954f-0913-4444-bf59-dcfd4158b72d/cluster-monitoring-operator/0.log" Jul 16 17:13:08.265134 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:08.265107 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wsdw_b40a9b93-01a7-422f-b912-6b9c08a6d64e/node-exporter/0.log" Jul 16 17:13:08.319215 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:08.319123 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wsdw_b40a9b93-01a7-422f-b912-6b9c08a6d64e/kube-rbac-proxy/0.log" Jul 16 17:13:08.402239 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:08.402208 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wsdw_b40a9b93-01a7-422f-b912-6b9c08a6d64e/init-textfile/0.log" Jul 16 17:13:10.781791 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:10.781763 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-tjjpr_4cc34f48-be5e-40c9-8afe-016ef627fda0/networking-console-plugin/0.log" Jul 16 17:13:10.998274 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:10.998232 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj"] Jul 16 17:13:11.017788 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.017144 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj"] Jul 16 17:13:11.017788 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.017346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.089610 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.089518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-proc\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.089919 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.089895 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdxwq\" (UniqueName: \"kubernetes.io/projected/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-kube-api-access-fdxwq\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.090072 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.090041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-podres\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.090195 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.090181 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-lib-modules\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.090345 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.090333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-sys\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.191764 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-podres\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.191764 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-lib-modules\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192008 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-sys\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192008 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-proc\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192008 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191879 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdxwq\" (UniqueName: \"kubernetes.io/projected/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-kube-api-access-fdxwq\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192008 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191932 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-podres\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192008 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.191975 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-lib-modules\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192330 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.192008 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-sys\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.192330 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.192040 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-proc\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.201352 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.201304 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdxwq\" (UniqueName: \"kubernetes.io/projected/39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26-kube-api-access-fdxwq\") pod \"perf-node-gather-daemonset-z42fj\" (UID: \"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26\") " pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.329548 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.329508 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.474899 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.474848 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/1.log" Jul 16 17:13:11.483346 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.483273 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6g8qg_cdfc70f4-b185-4c08-871b-fb69f667ea17/console-operator/2.log" Jul 16 17:13:11.529236 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.529138 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj"] Jul 16 17:13:11.864977 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.864941 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" event={"ID":"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26","Type":"ContainerStarted","Data":"2aa0438732e3005ce5bd583655ad554c5281e8c9f6272b4db04912a051f4d414"} Jul 16 17:13:11.865405 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.865001 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" event={"ID":"39ecbb5d-c4f8-42dc-bcd5-3f7fc67f8e26","Type":"ContainerStarted","Data":"4ac1a3120f0fc77d118d935388d01ea6e2d5ab775400f7adfa94d40968f51d03"} Jul 16 17:13:11.865405 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.865179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:11.890462 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:11.890396 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" podStartSLOduration=1.8903767930000002 podStartE2EDuration="1.890376793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:13:11.888870554 +0000 UTC m=+2867.696982569" watchObservedRunningTime="2026-07-16 17:13:11.890376793 +0000 UTC m=+2867.698488806" Jul 16 17:13:12.077426 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:12.077338 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-gwrlf_774b6eef-91d7-4267-8f74-f538e73313d1/download-server/0.log" Jul 16 17:13:12.680291 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:12.680260 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-kmv9b_bde6c88c-8138-4506-8702-1f2ae7a478e5/volume-data-source-validator/0.log" Jul 16 17:13:13.773021 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:13.772992 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-67bqh_17e7d176-96bd-4e7b-a4bf-177fe59d7860/dns/0.log" Jul 16 17:13:13.797891 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:13.797852 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-67bqh_17e7d176-96bd-4e7b-a4bf-177fe59d7860/kube-rbac-proxy/0.log" Jul 16 17:13:13.946028 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:13.946001 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8h4f9_579023c8-e486-4414-98ab-42b1dbfc20c6/dns-node-resolver/0.log" Jul 16 17:13:14.577034 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:14.576988 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7zm9t_edee432b-41a0-4a35-924e-405d84cdb218/node-ca/0.log" Jul 16 17:13:15.871123 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:15.870993 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6747f69598-6gkbs_c8930eed-7dda-4ce3-a11f-3e9cc46f62ce/kube-auth-proxy/0.log" Jul 16 17:13:16.052105 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:16.052078 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fb97c79fd-4cvvx_2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87/router/0.log" Jul 16 17:13:16.780966 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:16.780936 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-66dgj_f0271013-b398-4270-8978-4a2e6a37e4b5/serve-healthcheck-canary/0.log" Jul 16 17:13:17.301945 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:17.301908 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-dlqfr_6675174b-9998-44d2-a629-89c04b7951a1/insights-operator/0.log" Jul 16 17:13:17.302830 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:17.302811 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-dlqfr_6675174b-9998-44d2-a629-89c04b7951a1/insights-operator/1.log" Jul 16 17:13:17.327580 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:17.327553 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2b84v_6c2547e7-6a36-4bcb-bfbf-f9422a5c0527/kube-rbac-proxy/0.log" Jul 16 17:13:17.355785 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:17.355762 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2b84v_6c2547e7-6a36-4bcb-bfbf-f9422a5c0527/exporter/0.log" Jul 16 17:13:17.380925 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:17.380899 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2b84v_6c2547e7-6a36-4bcb-bfbf-f9422a5c0527/extractor/0.log" Jul 16 17:13:17.887120 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:17.887083 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-84tj4/perf-node-gather-daemonset-z42fj" Jul 16 17:13:20.193245 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:20.193206 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6847bf9b68-kxdlt_bb35eee2-eba0-42bb-97e7-90dfd9b170ac/manager/0.log" Jul 16 17:13:20.299524 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:20.299441 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dfhzq_7a768f94-dae6-4570-99ea-87b24a7c3dd4/manager/0.log" Jul 16 17:13:20.327277 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:20.327241 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846bd4c774-bgllj_b25dc11b-50e7-49ec-8183-0bf7918b9ffb/manager/0.log" Jul 16 17:13:20.356668 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:20.356633 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846bd4c774-bgllj_b25dc11b-50e7-49ec-8183-0bf7918b9ffb/copy-manifests/0.log" Jul 16 17:13:21.881548 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:21.881504 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6697dd89c8-gwsj9_2774729f-db7d-491d-8a3c-2d0e265d9efa/manager/0.log" Jul 16 17:13:21.934766 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:21.934733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rq5qd_b6ed089f-0c77-419d-83c1-49abb8202f92/openshift-lws-operator/0.log" Jul 16 17:13:22.127266 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:22.127237 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6747f69598-6gkbs_c8930eed-7dda-4ce3-a11f-3e9cc46f62ce/kube-auth-proxy/0.log" Jul 16 17:13:22.273453 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:22.273424 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fb97c79fd-4cvvx_2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87/router/0.log" Jul 16 17:13:23.816141 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:23.816036 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6747f69598-6gkbs_c8930eed-7dda-4ce3-a11f-3e9cc46f62ce/kube-auth-proxy/0.log" Jul 16 17:13:23.977636 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:23.977605 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fb97c79fd-4cvvx_2e2c29ca-b4a5-43cb-ba1d-e7b8e98a8f87/router/0.log" Jul 16 17:13:27.815342 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:27.815306 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2ks42_bf714cea-2ff9-4200-ad09-6c3f7eba5ae4/kube-storage-version-migrator-operator/1.log" Jul 16 17:13:27.816285 ip-10-0-131-94 kubenswrapper[2560]: I0716 17:13:27.816264 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2ks42_bf714cea-2ff9-4200-ad09-6c3f7eba5ae4/kube-storage-version-migrator-operator/0.log"