Jul 15 18:16:57.810906 ip-10-0-130-233 systemd[1]: Starting Kubernetes Kubelet... Jul 15 18:16:58.237244 ip-10-0-130-233 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 18:16:58.237244 ip-10-0-130-233 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 18:16:58.237244 ip-10-0-130-233 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 18:16:58.237244 ip-10-0-130-233 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 18:16:58.237244 ip-10-0-130-233 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 18:16:58.237244 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.230998 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239282 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239299 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239303 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239308 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239311 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239313 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239316 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239319 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239322 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239326 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239331 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239334 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239337 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239340 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239342 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239345 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239348 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:16:58.239373 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239350 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239353 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239355 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239358 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239360 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239363 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239365 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239368 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239370 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239373 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239376 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239383 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239385 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239388 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239391 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239393 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239396 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239398 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239401 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239403 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:16:58.239922 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239406 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239408 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239411 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239414 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239416 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239418 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239421 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239424 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239426 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239430 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239433 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239436 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239439 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239441 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239444 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239447 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239450 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239452 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239455 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239457 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:16:58.240478 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239461 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239463 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239465 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239468 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239471 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239473 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239476 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239478 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239480 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239483 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239485 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239488 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239490 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239493 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239497 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239500 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239503 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239506 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239509 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239511 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:16:58.241034 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239514 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239516 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239519 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239521 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239524 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239527 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239531 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239534 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239537 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239540 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239542 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239545 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239547 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239550 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239553 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239555 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239558 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239560 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239563 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239566 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:16:58.241534 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239569 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.239571 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240295 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240303 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240306 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240309 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240311 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240314 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240316 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240319 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240322 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240324 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240327 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240330 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240332 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240336 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240338 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240342 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240344 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:16:58.242143 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240347 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240350 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240353 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240355 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240358 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240360 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240363 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240365 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240368 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240370 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240373 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240375 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240378 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240381 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240383 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240386 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240389 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240391 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240394 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240396 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:16:58.242807 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240399 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240402 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240404 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240407 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240409 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240411 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240414 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240416 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240419 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240421 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240425 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240428 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240432 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240435 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240437 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240440 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240443 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240446 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240449 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240452 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:16:58.243365 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240454 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240457 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240459 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240462 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240464 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240467 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240470 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240472 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240475 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240478 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240480 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240483 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240486 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240489 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240491 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240494 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240497 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240499 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240502 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240505 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:16:58.243913 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240509 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240512 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240515 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240518 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240521 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240524 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240531 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240533 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240537 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240540 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240542 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240545 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240547 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240550 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240552 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240555 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240557 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240560 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240562 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240565 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:16:58.406096 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240568 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:16:58.313281 ip-10-0-130-233 systemd[1]: Started Kubernetes Kubelet. Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240572 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240977 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240982 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240986 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240989 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240992 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240994 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240997 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.240999 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241002 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241005 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241008 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241011 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241013 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241016 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241019 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241021 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241025 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:16:58.407228 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241027 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241030 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241033 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241035 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241038 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241042 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241046 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241049 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241051 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241055 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241059 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241062 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241065 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241068 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241072 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241075 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241078 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241081 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:16:58.408428 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241084 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241086 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241090 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241093 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241095 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241098 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241100 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241103 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241106 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241108 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241111 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241113 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241116 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241118 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241129 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241132 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241135 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241137 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241140 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241143 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241145 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:16:58.409104 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241148 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241151 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241153 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241156 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241158 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241162 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241164 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241167 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241169 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241172 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241175 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241177 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241181 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241184 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241186 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241190 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241193 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241196 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241199 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241201 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:16:58.409771 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241204 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241207 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241210 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241212 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241215 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241217 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241220 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241223 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241225 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241228 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241230 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241233 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241235 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241238 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241240 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241243 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241246 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241249 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241252 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241254 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:16:58.410405 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241257 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241260 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241262 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241340 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241350 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241360 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241367 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241376 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241382 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241388 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241393 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241396 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241399 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241403 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241406 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241409 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241412 2557 flags.go:64] FLAG: --cgroup-root="" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241415 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241418 2557 flags.go:64] FLAG: --client-ca-file="" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241422 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241425 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241430 2557 flags.go:64] FLAG: --cluster-domain="" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241432 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241436 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 18:16:58.411026 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241439 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241442 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241446 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241450 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241453 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241456 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241459 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241462 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241465 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241468 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241471 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241475 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241478 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241481 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241484 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241489 2557 flags.go:64] FLAG: --enable-server="true" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241492 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241497 2557 flags.go:64] FLAG: --event-burst="100" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241500 2557 flags.go:64] FLAG: --event-qps="50" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241503 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241506 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241509 2557 flags.go:64] FLAG: --eviction-hard="" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241513 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241516 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241519 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 18:16:59.898100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241522 2557 flags.go:64] FLAG: --eviction-soft="" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241525 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241528 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241532 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241535 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241538 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241541 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241544 2557 flags.go:64] FLAG: --feature-gates="" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241549 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241552 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241555 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241558 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241561 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241564 2557 flags.go:64] FLAG: --help="false" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241567 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-130-233.ec2.internal" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241570 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241573 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241576 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241579 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241583 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241586 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241588 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241592 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 18:16:59.899240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241594 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241597 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241600 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241616 2557 flags.go:64] FLAG: --kube-reserved="" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241619 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241622 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241625 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241628 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241630 2557 flags.go:64] FLAG: --lock-file="" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241633 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241636 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241639 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241644 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241648 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241651 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241653 2557 flags.go:64] FLAG: --logging-format="text" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241656 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241659 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241662 2557 flags.go:64] FLAG: --manifest-url="" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241665 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241674 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241677 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241681 2557 flags.go:64] FLAG: --max-pods="110" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241685 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241688 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 18:16:59.900231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241690 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241693 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241696 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241699 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241702 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241709 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241713 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241716 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241719 2557 flags.go:64] FLAG: --pod-cidr="" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241722 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241727 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241730 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241733 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241736 2557 flags.go:64] FLAG: --port="10250" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241739 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241742 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d34e6911b0e05cab" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241745 2557 flags.go:64] FLAG: --qos-reserved="" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241748 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241751 2557 flags.go:64] FLAG: --register-node="true" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241754 2557 flags.go:64] FLAG: --register-with-taints="" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241757 2557 flags.go:64] FLAG: --registry-burst="10" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241762 2557 flags.go:64] FLAG: --registry-qps="5" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241765 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241768 2557 flags.go:64] FLAG: --reserved-memory="" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241771 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 18:16:59.901113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241774 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241777 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241780 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241783 2557 flags.go:64] FLAG: --runonce="false" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241786 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241789 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241793 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241796 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241799 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241801 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241805 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241808 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241810 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241813 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241816 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241819 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241822 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241825 2557 flags.go:64] FLAG: --system-cgroups="" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241827 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241833 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241835 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241838 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241843 2557 flags.go:64] FLAG: --tls-min-version="" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241845 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241848 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 18:16:59.903332 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241851 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241854 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241857 2557 flags.go:64] FLAG: --v="2" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241861 2557 flags.go:64] FLAG: --version="false" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241866 2557 flags.go:64] FLAG: --vmodule="" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241870 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.241873 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241990 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241994 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.241998 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242001 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242004 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242007 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242009 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242012 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242015 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242017 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242020 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242023 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242025 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242029 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:16:59.904446 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242033 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242036 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242039 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242041 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242044 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242049 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242051 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242054 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242056 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242059 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242063 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242066 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242068 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242071 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242073 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242076 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242079 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242082 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242084 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242087 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:16:59.905489 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242091 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242094 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242099 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242102 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242104 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242107 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242110 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242112 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242115 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242117 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242120 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242122 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242125 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242127 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242130 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242132 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242135 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242139 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242142 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242145 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:16:59.906557 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242148 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242150 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242153 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242156 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242158 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242161 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242163 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242166 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242168 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242171 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242174 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242176 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242179 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242182 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242186 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242189 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242192 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242194 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242197 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242199 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:16:59.908076 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242202 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242204 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242207 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242210 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242212 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242214 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242218 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242222 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242225 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242229 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242232 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242235 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242238 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242240 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242243 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242246 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242248 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242258 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242261 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:16:59.908955 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242264 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242266 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242269 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242272 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242275 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.242277 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.242953 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.250289 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.250329 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.250891 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.250898 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250964 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250970 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250973 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250976 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:16:59.910449 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250979 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250982 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250985 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250987 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250990 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250995 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.250999 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251003 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251006 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251009 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251012 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251014 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251017 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251020 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251022 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251025 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251027 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251030 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251032 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:16:59.911308 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251034 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251037 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251040 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251044 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251048 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251051 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251059 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251063 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251066 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251069 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251071 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251074 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251076 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251079 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251082 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251084 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251087 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251089 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251092 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:16:59.912599 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251095 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251098 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251100 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251103 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251105 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251108 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251110 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251113 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251115 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251118 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251120 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251123 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251126 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251128 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251131 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251133 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251136 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251139 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251141 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251144 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:16:59.913460 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251155 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251158 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251161 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251164 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251167 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251169 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251172 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251175 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251177 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251180 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251182 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251185 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251188 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251191 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251193 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251196 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251199 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251201 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251204 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251206 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:16:59.914764 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251208 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251211 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251214 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251217 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251221 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251224 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251226 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251229 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251232 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251235 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251237 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251239 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251242 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251252 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251255 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251257 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:16:59.915646 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251260 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.251265 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251414 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251419 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251422 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251424 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251428 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251431 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251434 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251437 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251440 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251442 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251445 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251448 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251450 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 18:16:59.917168 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251453 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251455 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251458 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251460 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251463 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251465 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251468 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251470 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251473 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251476 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251478 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251481 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251483 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251486 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251488 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251496 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251499 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251502 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251504 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 18:16:59.918050 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251507 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251510 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251514 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251517 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251522 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251525 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251527 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251530 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251532 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251535 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251538 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251540 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251542 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251546 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251549 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251552 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251555 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251558 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251561 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 18:16:59.919350 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251564 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251566 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251569 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251571 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251574 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251576 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251579 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251582 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251585 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251589 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251599 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251615 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251619 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251621 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251624 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251626 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251629 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251635 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251637 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251640 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 18:16:59.920248 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251642 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251645 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251648 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251650 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251653 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251655 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251658 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251660 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251663 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251665 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251668 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251671 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251674 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251676 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251679 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251682 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251684 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251687 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251689 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251692 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 18:16:59.921139 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251694 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251698 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251701 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251710 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251712 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251715 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251717 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:16:58.251720 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.251725 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.251860 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.254153 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.255690 2557 server.go:1019] "Starting client certificate rotation" Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.255831 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 18:16:59.922023 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.255868 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.258073 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.261616 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.274455 2557 log.go:25] "Validated CRI v1 runtime API" Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.276964 2557 log.go:25] "Validated CRI v1 image API" Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.277717 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.279700 2557 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 98ed1492-1a99-4dfc-8c67-45028cbbd0aa:/dev/nvme0n1p4 ad74d68e-bba1-4e43-b946-a0f0bc28c586:/dev/nvme0n1p3] Jul 15 18:16:59.922675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.279715 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.285293 2557 manager.go:216] Machine: {Timestamp:2026-07-15 18:16:58.283396821 +0000 UTC m=+0.364372959 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100784 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f03a92701df694554bc048c807afb SystemUUID:ec2f03a9-2701-df69-4554-bc048c807afb BootID:f39f5741-a6bd-4d75-9544-11242f7f6961 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:53:7f:52:6c:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:53:7f:52:6c:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:03:36:58:92:cb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.285406 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.285506 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.288553 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.288580 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.289878 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.289889 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.289897 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.289934 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.290619 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.292237 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295406 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295564 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295593 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295616 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295639 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 18:16:59.923320 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295652 2557 kubelet.go:398] "Adding apiserver pod source" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.295674 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.296112 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.296129 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.298182 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.304632 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.304661 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305427 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305442 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305449 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305455 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305465 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305470 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305479 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305484 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305491 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305498 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305507 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.305512 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.306300 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.306307 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.306944 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.306966 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.309355 2557 server.go:1268] "Started kubelet" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.309455 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.309437 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.312725 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.314106 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.315000 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.317704 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.318914 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.320075 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lb5fv" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.322244 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.322899 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.322913 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.322924 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.323003 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.323162 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.322854 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.327645 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lb5fv" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.322275 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328197 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328209 2557 factory.go:55] Registering systemd factory Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328215 2557 factory.go:223] Registration of the systemd container factory successfully Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328406 2557 factory.go:153] Registering CRI-O factory Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328417 2557 factory.go:223] Registration of the crio container factory successfully Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328441 2557 factory.go:103] Registering Raw factory Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.328451 2557 manager.go:1196] Started watching for new ooms in manager Jul 15 18:16:59.924193 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.329068 2557 manager.go:349] Starting recovery of all containers Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.329801 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.332039 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.332088 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.331949 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-233.ec2.internal.18c2895922197847 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-233.ec2.internal,UID:ip-10-0-130-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-233.ec2.internal,},FirstTimestamp:2026-07-15 18:16:58.309326919 +0000 UTC m=+0.390303058,LastTimestamp:2026-07-15 18:16:58.309326919 +0000 UTC m=+0.390303058,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-233.ec2.internal,}" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.338808 2557 manager.go:354] Recovery completed Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.343185 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.345098 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.345134 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.345150 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346075 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346087 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346089 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346104 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346931 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346951 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.346961 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.347133 2557 policy_none.go:49] "None policy: Start" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.347144 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.347154 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.348102 2557 policy_none.go:47] "Start" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399213 2557 manager.go:337] "Starting Device Plugin manager" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.399384 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399394 2557 server.go:87] "Starting device plugin registration server" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399642 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399652 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399761 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399858 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:58.399867 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.400529 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 18:16:59.928037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:58.400567 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.971261 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.972268 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.972296 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.972306 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.972331 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.980172 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:59.980192 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-233.ec2.internal\": node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.991315 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.991348 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.991367 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:59.991395 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:16:59.993726 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:17:00.032297 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:16:59.996187 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.092257 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal"] Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.092345 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.093408 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.093446 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.093461 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.094785 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.094852 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.094876 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.095548 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.095576 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.095551 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.095589 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.095608 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.095623 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.096312 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.096519 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.096539 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.097141 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientMemory" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.097162 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.097172 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeHasSufficientPID" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.129501 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-233.ec2.internal\" not found" node="ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.137995 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-233.ec2.internal\" not found" node="ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.161229 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27d6e657e0fc42c84018a1763a94d169-config\") pod \"kube-apiserver-proxy-ip-10-0-130-233.ec2.internal\" (UID: \"27d6e657e0fc42c84018a1763a94d169\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.236682 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.196568 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.261776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27d6e657e0fc42c84018a1763a94d169-config\") pod \"kube-apiserver-proxy-ip-10-0-130-233.ec2.internal\" (UID: \"27d6e657e0fc42c84018a1763a94d169\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.261817 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a4630631040b00b5b7262f1ad99b11e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal\" (UID: \"4a4630631040b00b5b7262f1ad99b11e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.261837 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a4630631040b00b5b7262f1ad99b11e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal\" (UID: \"4a4630631040b00b5b7262f1ad99b11e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.261897 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27d6e657e0fc42c84018a1763a94d169-config\") pod \"kube-apiserver-proxy-ip-10-0-130-233.ec2.internal\" (UID: \"27d6e657e0fc42c84018a1763a94d169\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.297270 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.362511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a4630631040b00b5b7262f1ad99b11e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal\" (UID: \"4a4630631040b00b5b7262f1ad99b11e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.362540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a4630631040b00b5b7262f1ad99b11e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal\" (UID: \"4a4630631040b00b5b7262f1ad99b11e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.362588 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a4630631040b00b5b7262f1ad99b11e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal\" (UID: \"4a4630631040b00b5b7262f1ad99b11e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.373507 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.362630 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a4630631040b00b5b7262f1ad99b11e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal\" (UID: \"4a4630631040b00b5b7262f1ad99b11e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.495040 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.397783 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.495040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.431965 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.495040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.441540 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.659486 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.498873 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.659486 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.599361 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.699810 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-233.ec2.internal\" not found" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.722625 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.726367 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.726473 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.726507 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.767799 2557 apiserver.go:52] "Watching apiserver" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.791341 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.793696 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-t67r6","openshift-image-registry/node-ca-xd77m","openshift-network-diagnostics/network-check-target-h9rmb","openshift-network-operator/iptables-alerter-kzmms","openshift-ovn-kubernetes/ovnkube-node-dlr77","openshift-dns/node-resolver-56pv4","openshift-multus/multus-96qbj","openshift-multus/multus-additional-cni-plugins-82tdr","openshift-multus/network-metrics-daemon-csskv","kube-system/global-pull-secret-syncer-2n6fz","kube-system/konnectivity-agent-tvz6c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j"] Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.794482 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.795140 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.796358 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.798462 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.798743 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wvqfk\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.798791 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.799714 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.799808 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.799835 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.799913 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hppx4\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.799912 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.800145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.800218 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.800268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.801146 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.802270 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.802338 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 18:11:59 +0000 UTC" deadline="2027-12-25 16:03:28.591877601 +0000 UTC" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.802381 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12669h46m27.789500888s" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.803364 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.806683 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.806705 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nhwdl\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.806783 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.806790 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.819482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.807846 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808082 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hvn8q\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808128 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808163 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808342 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808362 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808398 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808421 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4hnhh\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.808398 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809122 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809138 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809256 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809304 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809317 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809350 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809359 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809382 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809433 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809478 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ld77f\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809712 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.809762 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.809963 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.810248 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6rc5f\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.810715 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.811052 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:00.811140 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:00.821299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.812205 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:00.827806 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.827762 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal"] Jul 15 18:17:00.827987 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.827965 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.829733 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.828410 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 18:17:00.829733 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.828476 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 18:17:00.829733 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.829261 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 18:17:00.829733 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.829593 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gnk7l\"" Jul 15 18:17:00.831002 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.830269 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal"] Jul 15 18:17:00.831002 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.830946 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 18:17:00.831577 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.831541 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 18:17:00.831832 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.831813 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pndnw\"" Jul 15 18:17:00.831924 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.831857 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 18:17:00.831975 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.831928 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 18:17:00.833275 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.833258 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rfllm" Jul 15 18:17:00.842353 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.842331 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rfllm" Jul 15 18:17:00.863845 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863827 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-modprobe-d\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.863934 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863852 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysconfig\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.863934 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863867 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysctl-conf\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.863934 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-tuned\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.863934 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863909 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-tmp\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.863934 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863931 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysctl-d\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-sys\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-kubernetes\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.863997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-run\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.864011 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-lib-modules\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.864027 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-systemd\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.864041 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpb7k\" (UniqueName: \"kubernetes.io/projected/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-kube-api-access-dpb7k\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.864064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-var-lib-kubelet\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.864256 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.864078 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-host\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.902367 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.902345 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 18:17:00.964675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964654 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-log-socket\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.964826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964681 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-device-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.964826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964698 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcgkp\" (UniqueName: \"kubernetes.io/projected/b639984f-f9b6-454a-a3e3-fd404870b7b7-kube-api-access-fcgkp\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.964826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964800 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-cni-bin\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.964826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-sys-fs\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-systemd\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-netns\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964909 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m7nt\" (UniqueName: \"kubernetes.io/projected/5bdcafe2-1ea6-4997-bf00-8471961ef963-kube-api-access-5m7nt\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964928 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-systemd\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964938 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-node-log\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.964977 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-cni-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965005 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965029 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-slash\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965044 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-run-netns\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965090 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19672c70-677e-46a2-9a56-399fc656d1a5-dbus\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-kubernetes\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-cnibin\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965160 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965165 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-kubernetes\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965191 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19672c70-677e-46a2-9a56-399fc656d1a5-kubelet-config\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpb7k\" (UniqueName: \"kubernetes.io/projected/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-kube-api-access-dpb7k\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-os-release\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965312 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-systemd\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965372 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-ovnkube-script-lib\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965395 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-system-cni-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965446 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-tmp\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965472 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-os-release\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965495 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-hostroot\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965517 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-conf-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.965613 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-sys\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965562 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-tmp-dir\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965630 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-sys\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965663 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-run\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965682 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eee677bf-a15f-47fb-911c-5003caa99181-host-slash\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965699 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2wfr\" (UniqueName: \"kubernetes.io/projected/19430ccd-cec5-413d-a155-58a1049530be-kube-api-access-j2wfr\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-system-cni-dir\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965727 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965747 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-run\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965758 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xdsd\" (UniqueName: \"kubernetes.io/projected/97944bdc-1dea-4012-ac18-4827adcd2da6-kube-api-access-9xdsd\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965814 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-cni-netd\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965829 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-cni-binary-copy\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-cni-multus\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965860 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19430ccd-cec5-413d-a155-58a1049530be-host\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19430ccd-cec5-413d-a155-58a1049530be-serviceca\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:00.966268 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.965986 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-var-lib-kubelet\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-host\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-modprobe-d\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966056 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-var-lib-kubelet\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysconfig\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysctl-conf\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966094 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-host\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966110 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysconfig\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966138 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-tuned\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysctl-d\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-socket-dir-parent\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966178 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-modprobe-d\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966208 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-k8s-cni-cncf-io\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966230 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysctl-conf\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-etc-kubernetes\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966273 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-cnibin\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966294 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-tuning-conf-dir\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.966866 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966294 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-sysctl-d\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966315 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-etc-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966392 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-ovn\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-lib-modules\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966476 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rqph\" (UniqueName: \"kubernetes.io/projected/eee677bf-a15f-47fb-911c-5003caa99181-kube-api-access-6rqph\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966501 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-hosts-file\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966501 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-lib-modules\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-kubelet\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966628 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rd98\" (UniqueName: \"kubernetes.io/projected/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-kube-api-access-9rd98\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966671 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-ovnkube-config\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966691 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-env-overrides\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966713 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clwcj\" (UniqueName: \"kubernetes.io/projected/18c21857-b3f7-4582-afea-13f190c60530-kube-api-access-clwcj\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966757 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdsrr\" (UniqueName: \"kubernetes.io/projected/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-kube-api-access-zdsrr\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966795 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-daemon-config\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-multus-certs\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.967678 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966853 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18c21857-b3f7-4582-afea-13f190c60530-ovn-node-metrics-cert\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.966878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967467 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bb98a3f-e75a-4b4a-a92f-9bb9feed6542-konnectivity-ca\") pod \"konnectivity-agent-tvz6c\" (UID: \"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542\") " pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967518 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-systemd-units\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967550 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eee677bf-a15f-47fb-911c-5003caa99181-iptables-alerter-script\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967582 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-cni-bin\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967614 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-var-lib-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967644 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bb98a3f-e75a-4b4a-a92f-9bb9feed6542-agent-certs\") pod \"konnectivity-agent-tvz6c\" (UID: \"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542\") " pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967676 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-socket-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967744 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-registration-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967800 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-cni-binary-copy\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967830 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:00.968446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.967878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-kubelet\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:00.970493 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.969195 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-etc-tuned\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.970938 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.970916 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-tmp\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:00.972977 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:00.972956 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpb7k\" (UniqueName: \"kubernetes.io/projected/ba681873-5e0f-4ce5-b135-ee2ae70ff09b-kube-api-access-dpb7k\") pod \"tuned-t67r6\" (UID: \"ba681873-5e0f-4ce5-b135-ee2ae70ff09b\") " pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:01.068644 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-os-release\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.068644 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-hostroot\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-conf-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-tmp-dir\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eee677bf-a15f-47fb-911c-5003caa99181-host-slash\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068736 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-hostroot\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068744 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eee677bf-a15f-47fb-911c-5003caa99181-host-slash\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068758 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-conf-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068761 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-os-release\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068807 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2wfr\" (UniqueName: \"kubernetes.io/projected/19430ccd-cec5-413d-a155-58a1049530be-kube-api-access-j2wfr\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068845 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-system-cni-dir\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068869 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xdsd\" (UniqueName: \"kubernetes.io/projected/97944bdc-1dea-4012-ac18-4827adcd2da6-kube-api-access-9xdsd\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:01.068880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068885 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-cni-netd\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068918 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-cni-binary-copy\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068950 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068956 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-system-cni-dir\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068970 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-cni-netd\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.068975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-cni-multus\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069006 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069030 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19430ccd-cec5-413d-a155-58a1049530be-host\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19430ccd-cec5-413d-a155-58a1049530be-serviceca\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-socket-dir-parent\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069008 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-cni-multus\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-tmp-dir\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069124 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-k8s-cni-cncf-io\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069146 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19430ccd-cec5-413d-a155-58a1049530be-host\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069149 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-etc-kubernetes\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069168 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-k8s-cni-cncf-io\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.069391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069165 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-socket-dir-parent\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069181 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-etc-kubernetes\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-cnibin\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069221 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-tuning-conf-dir\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069237 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-etc-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069255 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-ovn\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069278 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rqph\" (UniqueName: \"kubernetes.io/projected/eee677bf-a15f-47fb-911c-5003caa99181-kube-api-access-6rqph\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069282 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-cnibin\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069294 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-hosts-file\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069310 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-kubelet\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069323 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-ovn\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069291 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-etc-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069349 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-kubelet\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069368 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-hosts-file\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-tuning-conf-dir\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069434 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rd98\" (UniqueName: \"kubernetes.io/projected/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-kube-api-access-9rd98\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069456 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-cni-binary-copy\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069463 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-ovnkube-config\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069487 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-env-overrides\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwcj\" (UniqueName: \"kubernetes.io/projected/18c21857-b3f7-4582-afea-13f190c60530-kube-api-access-clwcj\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069535 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069559 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdsrr\" (UniqueName: \"kubernetes.io/projected/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-kube-api-access-zdsrr\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069580 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-daemon-config\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069600 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19430ccd-cec5-413d-a155-58a1049530be-serviceca\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069871 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-multus-certs\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069943 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-env-overrides\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.069757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-multus-certs\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070019 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18c21857-b3f7-4582-afea-13f190c60530-ovn-node-metrics-cert\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070060 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-ovnkube-config\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070060 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-daemon-config\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bb98a3f-e75a-4b4a-a92f-9bb9feed6542-konnectivity-ca\") pod \"konnectivity-agent-tvz6c\" (UID: \"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542\") " pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070125 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-systemd-units\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.070139 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070144 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eee677bf-a15f-47fb-911c-5003caa99181-iptables-alerter-script\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070165 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-cni-bin\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.070928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-var-lib-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070198 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-systemd-units\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070209 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-var-lib-openvswitch\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.070207 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:01.570179518 +0000 UTC m=+2.179992688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bb98a3f-e75a-4b4a-a92f-9bb9feed6542-agent-certs\") pod \"konnectivity-agent-tvz6c\" (UID: \"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542\") " pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-var-lib-cni-bin\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070279 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-socket-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070325 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-registration-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070355 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-cni-binary-copy\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070380 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070403 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-registration-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-kubelet\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070434 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-socket-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070445 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-kubelet\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070446 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-log-socket\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.071702 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070485 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-log-socket\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070523 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-device-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070550 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcgkp\" (UniqueName: \"kubernetes.io/projected/b639984f-f9b6-454a-a3e3-fd404870b7b7-kube-api-access-fcgkp\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070563 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-device-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070603 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-cni-bin\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070652 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bb98a3f-e75a-4b4a-a92f-9bb9feed6542-konnectivity-ca\") pod \"konnectivity-agent-tvz6c\" (UID: \"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542\") " pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.070664 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070655 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-sys-fs\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070657 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-cni-bin\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.070724 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:01.57070996 +0000 UTC m=+2.180523129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070729 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eee677bf-a15f-47fb-911c-5003caa99181-iptables-alerter-script\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-netns\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070801 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070812 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-host-run-netns\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070839 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b639984f-f9b6-454a-a3e3-fd404870b7b7-sys-fs\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7nt\" (UniqueName: \"kubernetes.io/projected/5bdcafe2-1ea6-4997-bf00-8471961ef963-kube-api-access-5m7nt\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.072184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070882 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-node-log\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-cni-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070979 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-node-log\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.070978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-slash\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-run-netns\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19672c70-677e-46a2-9a56-399fc656d1a5-dbus\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071096 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-cnibin\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071101 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-slash\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071149 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19672c70-677e-46a2-9a56-399fc656d1a5-kubelet-config\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071147 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-multus-cni-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071176 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-os-release\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071201 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-systemd\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071212 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-cnibin\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071240 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-cni-binary-copy\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.072698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071252 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-run-netns\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071260 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-ovnkube-script-lib\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071290 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071298 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19672c70-677e-46a2-9a56-399fc656d1a5-dbus\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071323 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bdcafe2-1ea6-4997-bf00-8471961ef963-os-release\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071339 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071343 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-run-systemd\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-system-cni-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18c21857-b3f7-4582-afea-13f190c60530-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071419 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19672c70-677e-46a2-9a56-399fc656d1a5-kubelet-config\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071474 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-system-cni-dir\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bdcafe2-1ea6-4997-bf00-8471961ef963-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.071760 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18c21857-b3f7-4582-afea-13f190c60530-ovnkube-script-lib\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.072847 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18c21857-b3f7-4582-afea-13f190c60530-ovn-node-metrics-cert\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.073186 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.072869 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bb98a3f-e75a-4b4a-a92f-9bb9feed6542-agent-certs\") pod \"konnectivity-agent-tvz6c\" (UID: \"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542\") " pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:01.077323 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.076990 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:01.077323 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.077015 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:01.077323 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.077028 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:01.077323 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.077120 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:01.577092701 +0000 UTC m=+2.186905882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:01.079108 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.079083 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rqph\" (UniqueName: \"kubernetes.io/projected/eee677bf-a15f-47fb-911c-5003caa99181-kube-api-access-6rqph\") pod \"iptables-alerter-kzmms\" (UID: \"eee677bf-a15f-47fb-911c-5003caa99181\") " pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.079204 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.079083 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdsrr\" (UniqueName: \"kubernetes.io/projected/f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e-kube-api-access-zdsrr\") pod \"node-resolver-56pv4\" (UID: \"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e\") " pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.079244 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.079228 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xdsd\" (UniqueName: \"kubernetes.io/projected/97944bdc-1dea-4012-ac18-4827adcd2da6-kube-api-access-9xdsd\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:01.079579 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.079559 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwcj\" (UniqueName: \"kubernetes.io/projected/18c21857-b3f7-4582-afea-13f190c60530-kube-api-access-clwcj\") pod \"ovnkube-node-dlr77\" (UID: \"18c21857-b3f7-4582-afea-13f190c60530\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.079965 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.079944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m7nt\" (UniqueName: \"kubernetes.io/projected/5bdcafe2-1ea6-4997-bf00-8471961ef963-kube-api-access-5m7nt\") pod \"multus-additional-cni-plugins-82tdr\" (UID: \"5bdcafe2-1ea6-4997-bf00-8471961ef963\") " pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.080140 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.080123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2wfr\" (UniqueName: \"kubernetes.io/projected/19430ccd-cec5-413d-a155-58a1049530be-kube-api-access-j2wfr\") pod \"node-ca-xd77m\" (UID: \"19430ccd-cec5-413d-a155-58a1049530be\") " pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.080252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.080235 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcgkp\" (UniqueName: \"kubernetes.io/projected/b639984f-f9b6-454a-a3e3-fd404870b7b7-kube-api-access-fcgkp\") pod \"aws-ebs-csi-driver-node-zs95j\" (UID: \"b639984f-f9b6-454a-a3e3-fd404870b7b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.080913 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.080898 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rd98\" (UniqueName: \"kubernetes.io/projected/e877bfe5-3035-4a48-b6d5-7c321c02fcf7-kube-api-access-9rd98\") pod \"multus-96qbj\" (UID: \"e877bfe5-3035-4a48-b6d5-7c321c02fcf7\") " pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.095974 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.095944 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d6e657e0fc42c84018a1763a94d169.slice/crio-013ccb5038bb1f6b3880d04a0d3020c12629a23ed61fda0d4f25dbe99b1c9e68 WatchSource:0}: Error finding container 013ccb5038bb1f6b3880d04a0d3020c12629a23ed61fda0d4f25dbe99b1c9e68: Status 404 returned error can't find the container with id 013ccb5038bb1f6b3880d04a0d3020c12629a23ed61fda0d4f25dbe99b1c9e68 Jul 15 18:17:01.096185 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.096169 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4630631040b00b5b7262f1ad99b11e.slice/crio-acacabe31daf76c4d4a3838a120c71907b6fc76b8660f7942db482369e12ccb6 WatchSource:0}: Error finding container acacabe31daf76c4d4a3838a120c71907b6fc76b8660f7942db482369e12ccb6: Status 404 returned error can't find the container with id acacabe31daf76c4d4a3838a120c71907b6fc76b8660f7942db482369e12ccb6 Jul 15 18:17:01.100017 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.100002 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 18:17:01.135933 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.135897 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t67r6" Jul 15 18:17:01.145182 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.145161 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba681873_5e0f_4ce5_b135_ee2ae70ff09b.slice/crio-625e8a2f29673a775e2e545b649f2174ee6444c6e47cbc5c2a764c63c834e310 WatchSource:0}: Error finding container 625e8a2f29673a775e2e545b649f2174ee6444c6e47cbc5c2a764c63c834e310: Status 404 returned error can't find the container with id 625e8a2f29673a775e2e545b649f2174ee6444c6e47cbc5c2a764c63c834e310 Jul 15 18:17:01.151389 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.151368 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kzmms" Jul 15 18:17:01.159165 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.159135 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee677bf_a15f_47fb_911c_5003caa99181.slice/crio-a69cfd3f428e926b135e853e3d3a25a78a9ccdd448acfb29a48e8677178a3adb WatchSource:0}: Error finding container a69cfd3f428e926b135e853e3d3a25a78a9ccdd448acfb29a48e8677178a3adb: Status 404 returned error can't find the container with id a69cfd3f428e926b135e853e3d3a25a78a9ccdd448acfb29a48e8677178a3adb Jul 15 18:17:01.169073 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.169050 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xd77m" Jul 15 18:17:01.178199 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.178171 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19430ccd_cec5_413d_a155_58a1049530be.slice/crio-2e84379961585c4edc07bbb653ce45a1636d77b4422b6728144227e5d5049882 WatchSource:0}: Error finding container 2e84379961585c4edc07bbb653ce45a1636d77b4422b6728144227e5d5049882: Status 404 returned error can't find the container with id 2e84379961585c4edc07bbb653ce45a1636d77b4422b6728144227e5d5049882 Jul 15 18:17:01.181119 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.181100 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:01.189059 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.189037 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c21857_b3f7_4582_afea_13f190c60530.slice/crio-e6c52ac91dd28a85574f443f015f8a5983ac31e40df50b5e50b4a729efee3dcd WatchSource:0}: Error finding container e6c52ac91dd28a85574f443f015f8a5983ac31e40df50b5e50b4a729efee3dcd: Status 404 returned error can't find the container with id e6c52ac91dd28a85574f443f015f8a5983ac31e40df50b5e50b4a729efee3dcd Jul 15 18:17:01.210974 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.210950 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-56pv4" Jul 15 18:17:01.219425 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.219399 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ca71a2_f8c6_4eb3_8f5e_c5d0e12a9a3e.slice/crio-0a760e916aee9b545c8dc502a75b1358d50b51839b7afd701e7956271f4d9543 WatchSource:0}: Error finding container 0a760e916aee9b545c8dc502a75b1358d50b51839b7afd701e7956271f4d9543: Status 404 returned error can't find the container with id 0a760e916aee9b545c8dc502a75b1358d50b51839b7afd701e7956271f4d9543 Jul 15 18:17:01.222438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.222422 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-96qbj" Jul 15 18:17:01.230246 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.230223 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode877bfe5_3035_4a48_b6d5_7c321c02fcf7.slice/crio-4c4a5e0d4198fb350448fcd22fcb4ebe5a1d30e31b5e7fab318e72cc27429f5e WatchSource:0}: Error finding container 4c4a5e0d4198fb350448fcd22fcb4ebe5a1d30e31b5e7fab318e72cc27429f5e: Status 404 returned error can't find the container with id 4c4a5e0d4198fb350448fcd22fcb4ebe5a1d30e31b5e7fab318e72cc27429f5e Jul 15 18:17:01.230349 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.230257 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-82tdr" Jul 15 18:17:01.238714 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.238696 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bdcafe2_1ea6_4997_bf00_8471961ef963.slice/crio-3bdc55e029f6886c6662a594d94274ff86c9dd67f1fd9fd996e569efd0d80b61 WatchSource:0}: Error finding container 3bdc55e029f6886c6662a594d94274ff86c9dd67f1fd9fd996e569efd0d80b61: Status 404 returned error can't find the container with id 3bdc55e029f6886c6662a594d94274ff86c9dd67f1fd9fd996e569efd0d80b61 Jul 15 18:17:01.245484 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.245460 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:01.255677 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:01.255649 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb98a3f_e75a_4b4a_a92f_9bb9feed6542.slice/crio-9e112d5af6966d972aea05912f10074b2532f5e8800e30720abfbd640a86538a WatchSource:0}: Error finding container 9e112d5af6966d972aea05912f10074b2532f5e8800e30720abfbd640a86538a: Status 404 returned error can't find the container with id 9e112d5af6966d972aea05912f10074b2532f5e8800e30720abfbd640a86538a Jul 15 18:17:01.263928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.263910 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" Jul 15 18:17:01.276111 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.276093 2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-zs95j_openshift-cluster-csi-drivers(b639984f-f9b6-454a-a3e3-fd404870b7b7): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 15 18:17:01.276836 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.276818 2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-zs95j_openshift-cluster-csi-drivers(b639984f-f9b6-454a-a3e3-fd404870b7b7): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 15 18:17:01.278548 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.278531 2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-zs95j_openshift-cluster-csi-drivers(b639984f-f9b6-454a-a3e3-fd404870b7b7): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 15 18:17:01.278590 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.278573 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" podUID="b639984f-f9b6-454a-a3e3-fd404870b7b7" Jul 15 18:17:01.322605 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.322578 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:17:01.575248 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.575064 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.575248 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.575120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:01.575248 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.575230 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:01.575510 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.575282 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:01.575510 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.575305 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:02.575284954 +0000 UTC m=+3.185098133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:01.575510 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.575347 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:02.575323773 +0000 UTC m=+3.185136942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:01.675695 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.675653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:01.675922 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.675901 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:01.675985 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.675934 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:01.676032 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.676007 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:01.676082 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.676067 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:02.676047861 +0000 UTC m=+3.285861028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:01.844134 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.844003 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 18:12:00 +0000 UTC" deadline="2027-12-14 14:37:03.60331353 +0000 UTC" Jul 15 18:17:01.844134 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.844077 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12404h20m1.759241089s" Jul 15 18:17:01.885718 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.885683 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:17:01.992950 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.992915 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:01.993133 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.993063 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:01.993718 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:01.993695 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:01.993843 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:01.993818 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:02.005072 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.005013 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tvz6c" event={"ID":"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542","Type":"ContainerStarted","Data":"9e112d5af6966d972aea05912f10074b2532f5e8800e30720abfbd640a86538a"} Jul 15 18:17:02.016799 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.016744 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-96qbj" event={"ID":"e877bfe5-3035-4a48-b6d5-7c321c02fcf7","Type":"ContainerStarted","Data":"4c4a5e0d4198fb350448fcd22fcb4ebe5a1d30e31b5e7fab318e72cc27429f5e"} Jul 15 18:17:02.024353 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.024318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-56pv4" event={"ID":"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e","Type":"ContainerStarted","Data":"0a760e916aee9b545c8dc502a75b1358d50b51839b7afd701e7956271f4d9543"} Jul 15 18:17:02.028207 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.028111 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t67r6" event={"ID":"ba681873-5e0f-4ce5-b135-ee2ae70ff09b","Type":"ContainerStarted","Data":"625e8a2f29673a775e2e545b649f2174ee6444c6e47cbc5c2a764c63c834e310"} Jul 15 18:17:02.032455 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.032417 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" event={"ID":"27d6e657e0fc42c84018a1763a94d169","Type":"ContainerStarted","Data":"013ccb5038bb1f6b3880d04a0d3020c12629a23ed61fda0d4f25dbe99b1c9e68"} Jul 15 18:17:02.036192 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.036158 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" event={"ID":"4a4630631040b00b5b7262f1ad99b11e","Type":"ContainerStarted","Data":"acacabe31daf76c4d4a3838a120c71907b6fc76b8660f7942db482369e12ccb6"} Jul 15 18:17:02.040609 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.040566 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" event={"ID":"b639984f-f9b6-454a-a3e3-fd404870b7b7","Type":"ContainerStarted","Data":"2f4105160fcabbc2c1dadcfd8a4b029fcc6b3ae66100636453557fb656cdc644"} Jul 15 18:17:02.044038 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.044009 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerStarted","Data":"3bdc55e029f6886c6662a594d94274ff86c9dd67f1fd9fd996e569efd0d80b61"} Jul 15 18:17:02.048818 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.048763 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:23e38a7cf7c4e5c572a5461047f39d2f821e55490088f442a842f66da801ae99\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0512f4cb34792d3c7553c42a85869c34b5953550c019b958649e3246d6540d94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0f984e9f25655116e61604ac13881b82e2830789119e02b44a338cea95f30425\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" podUID="b639984f-f9b6-454a-a3e3-fd404870b7b7" Jul 15 18:17:02.056731 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.054250 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"e6c52ac91dd28a85574f443f015f8a5983ac31e40df50b5e50b4a729efee3dcd"} Jul 15 18:17:02.056886 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.056732 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xd77m" event={"ID":"19430ccd-cec5-413d-a155-58a1049530be","Type":"ContainerStarted","Data":"2e84379961585c4edc07bbb653ce45a1636d77b4422b6728144227e5d5049882"} Jul 15 18:17:02.060003 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.059977 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kzmms" event={"ID":"eee677bf-a15f-47fb-911c-5003caa99181","Type":"ContainerStarted","Data":"a69cfd3f428e926b135e853e3d3a25a78a9ccdd448acfb29a48e8677178a3adb"} Jul 15 18:17:02.481174 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.481146 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 18:17:02.585262 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.585227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:02.585437 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.585277 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:02.585437 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.585416 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:02.585548 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.585478 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:04.585459047 +0000 UTC m=+5.195272231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:02.585934 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.585890 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:02.586055 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.585941 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:04.585927088 +0000 UTC m=+5.195740254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:02.685844 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.685805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:02.686028 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.686017 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:02.686096 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.686041 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:02.686096 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.686054 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:02.686196 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.686113 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:04.686094916 +0000 UTC m=+5.295908090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:02.845201 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.845107 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 18:12:00 +0000 UTC" deadline="2028-01-22 14:59:55.259645228 +0000 UTC" Jul 15 18:17:02.845201 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.845152 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13340h42m52.414498433s" Jul 15 18:17:02.991837 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:02.991800 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:02.992020 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:02.991940 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:03.992944 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:03.992872 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:03.992944 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:03.992896 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:03.993440 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:03.993011 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:03.993502 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:03.993421 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:04.599590 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:04.599546 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:04.599788 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:04.599606 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:04.599788 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.599736 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:04.599909 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.599817 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:08.599796941 +0000 UTC m=+9.209610127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:04.599983 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.599901 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:04.599983 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.599966 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:08.599947788 +0000 UTC m=+9.209760957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:04.700623 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:04.700582 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:04.700836 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.700819 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:04.700908 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.700844 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:04.700908 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.700856 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:04.701007 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.700917 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:08.700897666 +0000 UTC m=+9.310710845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:04.992484 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:04.992405 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:04.992652 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:04.992542 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:05.993779 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:05.993727 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:05.994244 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:05.993908 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:05.994244 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:05.994072 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:05.994244 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:05.994172 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:06.992331 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:06.992255 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:06.992500 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:06.992385 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:07.991998 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:07.991916 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:07.992351 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:07.992050 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:07.992351 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:07.992113 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:07.992351 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:07.992195 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:08.634325 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:08.634283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:08.634511 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:08.634338 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:08.634511 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.634465 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:08.634511 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.634467 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:08.634678 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.634527 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:16.634506575 +0000 UTC m=+17.244319742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:08.634678 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.634546 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:16.634536785 +0000 UTC m=+17.244349959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:08.735692 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:08.735648 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:08.735912 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.735861 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:08.735912 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.735887 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:08.735912 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.735900 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:08.736031 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.735959 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:16.73594027 +0000 UTC m=+17.345753439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:08.992462 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:08.992381 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:08.992798 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:08.992527 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:09.993307 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:09.993224 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:09.993704 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:09.993352 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:09.993704 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:09.993697 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:09.993831 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:09.993811 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:10.993037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:10.992545 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:10.993037 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:10.992649 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:11.991878 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:11.991842 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:11.992337 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:11.991994 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:11.992337 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:11.992053 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:11.992337 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:11.992149 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:12.992570 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:12.992379 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:12.993025 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:12.992661 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:13.991864 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:13.991834 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:13.992025 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:13.991835 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:13.992025 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:13.991965 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:13.992149 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:13.992030 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:14.992062 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:14.992028 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:14.992500 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:14.992167 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:15.992707 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:15.992647 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:15.993152 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:15.992647 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:15.993152 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:15.992811 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:15.993152 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:15.992842 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:16.694375 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:16.694328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:16.694552 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:16.694389 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:16.694552 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.694447 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:16.694552 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.694521 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:32.694499566 +0000 UTC m=+33.304312748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:16.694552 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.694529 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:16.694716 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.694575 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:32.694560934 +0000 UTC m=+33.304374101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:16.795677 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:16.795639 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:16.795886 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.795848 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:16.795886 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.795873 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:16.795886 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.795887 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:16.796044 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.795951 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:32.795931712 +0000 UTC m=+33.405744897 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:16.991874 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:16.991782 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:16.992028 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:16.991923 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:17.992606 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:17.992567 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:17.993052 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:17.992692 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:17.993052 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:17.992758 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:17.993052 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:17.992872 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:18.992045 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:18.992010 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:18.992204 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:18.992137 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:19.993892 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:19.993864 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:19.994279 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:19.993983 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:19.994279 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:19.994027 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:19.994279 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:19.994154 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:20.992424 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:20.992269 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:20.992574 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:20.992550 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:21.105925 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.105855 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xd77m" event={"ID":"19430ccd-cec5-413d-a155-58a1049530be","Type":"ContainerStarted","Data":"68b4bb65d153e9ce751036dfa5a802615411939c8a21156711e955198e91a49f"} Jul 15 18:17:21.108828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.108216 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tvz6c" event={"ID":"1bb98a3f-e75a-4b4a-a92f-9bb9feed6542","Type":"ContainerStarted","Data":"674acd3aa20b8986dd2915a1c986477e57d9f4a30804ea2332a3ffb7c939a2be"} Jul 15 18:17:21.112329 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.111454 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-96qbj" event={"ID":"e877bfe5-3035-4a48-b6d5-7c321c02fcf7","Type":"ContainerStarted","Data":"b0ce75d21a0bbf387879960d190124d0b2eb0c1467916035b426fe68c7d5b79b"} Jul 15 18:17:21.113825 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.113688 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-56pv4" event={"ID":"f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e","Type":"ContainerStarted","Data":"bbcfc941f5a11b137fb3abecbbd25998524aa96bfde0e7df854fae0b78f88758"} Jul 15 18:17:21.115942 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.115915 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t67r6" event={"ID":"ba681873-5e0f-4ce5-b135-ee2ae70ff09b","Type":"ContainerStarted","Data":"7a3b3a2c8f4c34f492bdc1cf3a6f0a5d0188ea4e895c19d14becfe43cb62c3d0"} Jul 15 18:17:21.117361 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.117327 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" event={"ID":"27d6e657e0fc42c84018a1763a94d169","Type":"ContainerStarted","Data":"125ba9315c417a267cf382aac3a0d913079edd90abca3f8fd3b70dce3183cb74"} Jul 15 18:17:21.119303 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.119281 2557 generic.go:356] "Generic (PLEG): container finished" podID="4a4630631040b00b5b7262f1ad99b11e" containerID="7f12f1948047deff085871260622737a575e5ba9e645e1fc19ff1cba1310f0ce" exitCode=0 Jul 15 18:17:21.119365 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.119351 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" event={"ID":"4a4630631040b00b5b7262f1ad99b11e","Type":"ContainerDied","Data":"7f12f1948047deff085871260622737a575e5ba9e645e1fc19ff1cba1310f0ce"} Jul 15 18:17:21.120478 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.120441 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xd77m" podStartSLOduration=7.877215489 podStartE2EDuration="21.120429785s" totalImagesPullingTime="13.243214296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:21.120130783 +0000 UTC m=+21.729943971" watchObservedRunningTime="2026-07-15 18:17:21.120429785 +0000 UTC m=+21.730242972" Jul 15 18:17:21.121068 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.121024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" event={"ID":"b639984f-f9b6-454a-a3e3-fd404870b7b7","Type":"ContainerStarted","Data":"4c6c4ab680f010816c66ad23a04ec4e31d7b88d3bc23221a3fc237c7f32075ca"} Jul 15 18:17:21.122896 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.122872 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bdcafe2-1ea6-4997-bf00-8471961ef963" containerID="06cfc5bb0cd2b08560921adeebc82d382f168b251dad1891aa3e2a639639fc54" exitCode=0 Jul 15 18:17:21.122979 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.122960 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerDied","Data":"06cfc5bb0cd2b08560921adeebc82d382f168b251dad1891aa3e2a639639fc54"} Jul 15 18:17:21.125204 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.125179 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:17:21.125872 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.125851 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"ebd181c3bfefd25ed9aed74af94a297cf31f6675dd6abee1a8445d012cd0851e"} Jul 15 18:17:21.125956 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.125919 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"2d6745ea77b416682c9b0e85707a45843c31e478b30210f6a8ff0980c3edba51"} Jul 15 18:17:21.157573 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.155039 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t67r6" podStartSLOduration=3.312789404 podStartE2EDuration="22.155017337s" totalImagesPullingTime="18.842227933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:21.139038367 +0000 UTC m=+21.748851556" watchObservedRunningTime="2026-07-15 18:17:21.155017337 +0000 UTC m=+21.764830538" Jul 15 18:17:21.171673 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.171613 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tvz6c" podStartSLOduration=12.287740682999999 podStartE2EDuration="21.171589413s" totalImagesPullingTime="8.88384873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:21.158398183 +0000 UTC m=+21.768211372" watchObservedRunningTime="2026-07-15 18:17:21.171589413 +0000 UTC m=+21.781402602" Jul 15 18:17:21.172743 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.172585 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-233.ec2.internal" podStartSLOduration=21.172570433 podStartE2EDuration="21.172570433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:21.171542031 +0000 UTC m=+21.781355220" watchObservedRunningTime="2026-07-15 18:17:21.172570433 +0000 UTC m=+21.782383622" Jul 15 18:17:21.188148 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.187957 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-56pv4" podStartSLOduration=6.857698995 podStartE2EDuration="21.187934969s" totalImagesPullingTime="14.330235974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:21.187126238 +0000 UTC m=+21.796939430" watchObservedRunningTime="2026-07-15 18:17:21.187934969 +0000 UTC m=+21.797748246" Jul 15 18:17:21.206051 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.206006 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-96qbj" podStartSLOduration=2.41523122 podStartE2EDuration="21.205991297s" totalImagesPullingTime="18.790760077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:21.204213187 +0000 UTC m=+21.814026375" watchObservedRunningTime="2026-07-15 18:17:21.205991297 +0000 UTC m=+21.815804481" Jul 15 18:17:21.992396 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.992363 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:21.992657 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:21.992363 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:21.992657 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:21.992510 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:21.992657 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:21.992609 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:22.129406 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.129359 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" event={"ID":"4a4630631040b00b5b7262f1ad99b11e","Type":"ContainerStarted","Data":"9752f5b146218063b2d0b63a98c6198f9d3be5f9f84b556509d5f5734cfe9587"} Jul 15 18:17:22.132616 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.132588 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:17:22.133545 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.133520 2557 generic.go:356] "Generic (PLEG): container finished" podID="18c21857-b3f7-4582-afea-13f190c60530" containerID="ebd181c3bfefd25ed9aed74af94a297cf31f6675dd6abee1a8445d012cd0851e" exitCode=1 Jul 15 18:17:22.133654 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.133588 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerDied","Data":"ebd181c3bfefd25ed9aed74af94a297cf31f6675dd6abee1a8445d012cd0851e"} Jul 15 18:17:22.133701 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.133653 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"f0e50419781797a9ce88f5a9ce66e135fb5b414581785c3018d0014268515962"} Jul 15 18:17:22.133701 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.133692 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"06e0755078c2faff3eaae79e8bb13d902cfbe74f33e66ba136f2dd33dd1b91c1"} Jul 15 18:17:22.133798 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.133726 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"abdcabbdaf3d843316987b7918f3f33b7c605ada78290b18e9ca1bfb5c2e642a"} Jul 15 18:17:22.133798 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.133760 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"6527c8a8ee43d677e7d51eb0e83597d46070bc3fbe8977ec94ca9412bb180166"} Jul 15 18:17:22.135393 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.135365 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kzmms" event={"ID":"eee677bf-a15f-47fb-911c-5003caa99181","Type":"ContainerStarted","Data":"8ed22f1fb8b367e846ae794767defb5b15196ca806ac8cfc311807191ad453c7"} Jul 15 18:17:22.147965 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.147917 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-233.ec2.internal" podStartSLOduration=22.147900266 podStartE2EDuration="22.147900266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:22.146587562 +0000 UTC m=+22.756400751" watchObservedRunningTime="2026-07-15 18:17:22.147900266 +0000 UTC m=+22.757713456" Jul 15 18:17:22.160922 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.160884 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kzmms" podStartSLOduration=8.770388866 podStartE2EDuration="23.160871054s" totalImagesPullingTime="14.390482188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:22.160338327 +0000 UTC m=+22.770151518" watchObservedRunningTime="2026-07-15 18:17:22.160871054 +0000 UTC m=+22.770684266" Jul 15 18:17:22.284819 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.284789 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 18:17:22.886572 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.886468 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T18:17:22.284812201Z","UUID":"19ad1525-523b-4984-9480-2d3c7f3652b8","Handler":null,"Name":"","Endpoint":""} Jul 15 18:17:22.888092 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.888064 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 18:17:22.888200 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.888104 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 18:17:22.991814 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:22.991782 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:22.991998 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:22.991912 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:23.139560 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:23.139480 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" event={"ID":"b639984f-f9b6-454a-a3e3-fd404870b7b7","Type":"ContainerStarted","Data":"78c6e090acf28be4e72819d8af90e940221ba4bb755b6909953fbc29b4dc13b3"} Jul 15 18:17:23.996938 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:23.996707 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:23.997106 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:23.996748 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:23.997106 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:23.997053 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:23.997225 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:23.997118 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:24.144516 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:24.144480 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" event={"ID":"b639984f-f9b6-454a-a3e3-fd404870b7b7","Type":"ContainerStarted","Data":"9a8b53b91077cfa6a404bd1bcc7fba127bebf74e7f7eea543b811f24656e6b82"} Jul 15 18:17:24.166121 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:24.166065 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zs95j" podStartSLOduration=1.9691477530000001 podStartE2EDuration="24.166042587s" totalImagesPullingTime="22.196894834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-15 18:17:24.165263503 +0000 UTC m=+24.775076691" watchObservedRunningTime="2026-07-15 18:17:24.166042587 +0000 UTC m=+24.775855777" Jul 15 18:17:24.992292 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:24.992258 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:24.992482 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:24.992368 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:25.582979 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:25.582899 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:25.583517 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:25.583450 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:25.991977 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:25.991943 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:25.992168 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:25.992075 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:25.992168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:25.992120 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:25.992281 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:25.992209 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:26.150519 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.150479 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bdcafe2-1ea6-4997-bf00-8471961ef963" containerID="f69ef6a1851fb19c2531d377a500a6060b8f1df44856ce178976f32e07541f5b" exitCode=0 Jul 15 18:17:26.150674 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.150536 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerDied","Data":"f69ef6a1851fb19c2531d377a500a6060b8f1df44856ce178976f32e07541f5b"} Jul 15 18:17:26.153608 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.153590 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:17:26.153980 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.153954 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"c47cb36348d93cf6548f1a414864e057fe03d55857b7d278f4d7e06b411e2198"} Jul 15 18:17:26.603659 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.603631 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:26.604322 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.604301 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tvz6c" Jul 15 18:17:26.991594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:26.991554 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:26.991802 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:26.991679 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:27.158295 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:27.158202 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bdcafe2-1ea6-4997-bf00-8471961ef963" containerID="e3c76b9e8ee9c8222cb258e1620b4d6be16c6142382942bebf43af421f938071" exitCode=0 Jul 15 18:17:27.158295 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:27.158275 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerDied","Data":"e3c76b9e8ee9c8222cb258e1620b4d6be16c6142382942bebf43af421f938071"} Jul 15 18:17:27.998056 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:27.997872 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:27.998502 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:27.997945 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:27.998502 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:27.998139 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:27.998502 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:27.998197 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:28.162223 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:28.162184 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bdcafe2-1ea6-4997-bf00-8471961ef963" containerID="fd66672db02fd2fd6c2ad9d7c89be318d6cb8912989bff6a1615233661f177c6" exitCode=0 Jul 15 18:17:28.162383 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:28.162274 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerDied","Data":"fd66672db02fd2fd6c2ad9d7c89be318d6cb8912989bff6a1615233661f177c6"} Jul 15 18:17:28.165694 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:28.165678 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:17:28.166066 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:28.166042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"643a3ad56e6e85d50712703cc88cabf3506e66446d807a90bcf38df7b7ee330b"} Jul 15 18:17:28.166508 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:28.166492 2557 scope.go:117] "RemoveContainer" containerID="ebd181c3bfefd25ed9aed74af94a297cf31f6675dd6abee1a8445d012cd0851e" Jul 15 18:17:28.991987 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:28.991956 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:28.992205 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:28.992084 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:29.174826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.174800 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:17:29.175456 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.175418 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" event={"ID":"18c21857-b3f7-4582-afea-13f190c60530","Type":"ContainerStarted","Data":"f0c21c3bfd69590ff63b7c93095d9b2e9015aec78f98c02747553f3fdb6e6f7e"} Jul 15 18:17:29.175738 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.175717 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:29.175861 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.175804 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:29.175861 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.175851 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:29.193827 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.193603 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:29.194761 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.194738 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:17:29.205841 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.205798 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" podStartSLOduration=9.432042516 podStartE2EDuration="29.205780861s" totalImagesPullingTime="19.773738345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:29.20389686 +0000 UTC m=+29.813710049" watchObservedRunningTime="2026-07-15 18:17:29.205780861 +0000 UTC m=+29.815594067" Jul 15 18:17:29.232631 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.232598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h9rmb"] Jul 15 18:17:29.232806 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.232672 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2n6fz"] Jul 15 18:17:29.232806 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.232748 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:29.232931 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:29.232874 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:29.235679 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.235648 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-csskv"] Jul 15 18:17:29.235862 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.235847 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:29.236025 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:29.236004 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:29.238703 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:29.238551 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:29.238703 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:29.238667 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:30.992550 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:30.992506 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:30.992550 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:30.992538 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:30.993124 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:30.992548 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:30.993124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:30.992652 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:30.993124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:30.992720 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:30.993124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:30.992831 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:32.721118 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:32.721082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:32.721118 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:32.721121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:32.721590 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.721222 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:32.721590 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.721224 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:32.721590 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.721266 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs podName:97944bdc-1dea-4012-ac18-4827adcd2da6 nodeName:}" failed. No retries permitted until 2026-07-15 18:18:04.721254983 +0000 UTC m=+65.331068149 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs") pod "network-metrics-daemon-csskv" (UID: "97944bdc-1dea-4012-ac18-4827adcd2da6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 18:17:32.721590 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.721279 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret podName:19672c70-677e-46a2-9a56-399fc656d1a5 nodeName:}" failed. No retries permitted until 2026-07-15 18:18:04.721273222 +0000 UTC m=+65.331086388 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret") pod "global-pull-secret-syncer-2n6fz" (UID: "19672c70-677e-46a2-9a56-399fc656d1a5") : object "kube-system"/"original-pull-secret" not registered Jul 15 18:17:32.822102 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:32.822065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:32.822265 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.822234 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 18:17:32.822265 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.822261 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 18:17:32.822346 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.822273 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9cm7l for pod openshift-network-diagnostics/network-check-target-h9rmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:32.822346 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.822329 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l podName:6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1 nodeName:}" failed. No retries permitted until 2026-07-15 18:18:04.822310555 +0000 UTC m=+65.432123721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9cm7l" (UniqueName: "kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l") pod "network-check-target-h9rmb" (UID: "6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 18:17:32.992047 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:32.991986 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:32.992178 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:32.992069 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:32.992178 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.992073 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2n6fz" podUID="19672c70-677e-46a2-9a56-399fc656d1a5" Jul 15 18:17:32.992178 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.992169 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h9rmb" podUID="6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1" Jul 15 18:17:32.992296 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:32.992192 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:32.992296 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:32.992246 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csskv" podUID="97944bdc-1dea-4012-ac18-4827adcd2da6" Jul 15 18:17:33.186870 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.186829 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bdcafe2-1ea6-4997-bf00-8471961ef963" containerID="ed96f2e124a173554a77c1ef6658c53629ee78df6f968900ac15d5f4460d97d8" exitCode=0 Jul 15 18:17:33.187031 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.186892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerDied","Data":"ed96f2e124a173554a77c1ef6658c53629ee78df6f968900ac15d5f4460d97d8"} Jul 15 18:17:33.696881 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.696854 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-233.ec2.internal" event="NodeReady" Jul 15 18:17:33.697048 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.696981 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 18:17:33.740448 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.740401 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xx5wd"] Jul 15 18:17:33.760996 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.760971 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-htk6w"] Jul 15 18:17:33.761142 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.761096 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.763588 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.763558 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-42x79\"" Jul 15 18:17:33.763709 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.763569 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 18:17:33.763750 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.763705 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 18:17:33.781051 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.781030 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-htk6w"] Jul 15 18:17:33.781157 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.781058 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xx5wd"] Jul 15 18:17:33.781157 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.781145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:33.783529 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.783507 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 18:17:33.783649 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.783577 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 18:17:33.783649 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.783604 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 18:17:33.783927 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.783912 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4vnzq\"" Jul 15 18:17:33.830315 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.830284 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34cf3357-ec56-4e44-8b1a-58379395479b-config-volume\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.830315 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.830313 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550040cc-16c3-4d74-80a0-908ee435260a-cert\") pod \"ingress-canary-htk6w\" (UID: \"550040cc-16c3-4d74-80a0-908ee435260a\") " pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:33.830486 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.830395 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cf3357-ec56-4e44-8b1a-58379395479b-tmp-dir\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.830486 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.830413 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqd7j\" (UniqueName: \"kubernetes.io/projected/34cf3357-ec56-4e44-8b1a-58379395479b-kube-api-access-fqd7j\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.830486 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.830438 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.830486 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.830461 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6fpv\" (UniqueName: \"kubernetes.io/projected/550040cc-16c3-4d74-80a0-908ee435260a-kube-api-access-w6fpv\") pod \"ingress-canary-htk6w\" (UID: \"550040cc-16c3-4d74-80a0-908ee435260a\") " pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:33.931455 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.931426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cf3357-ec56-4e44-8b1a-58379395479b-tmp-dir\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.931536 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.931460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqd7j\" (UniqueName: \"kubernetes.io/projected/34cf3357-ec56-4e44-8b1a-58379395479b-kube-api-access-fqd7j\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.931536 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.931519 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.931595 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.931563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6fpv\" (UniqueName: \"kubernetes.io/projected/550040cc-16c3-4d74-80a0-908ee435260a-kube-api-access-w6fpv\") pod \"ingress-canary-htk6w\" (UID: \"550040cc-16c3-4d74-80a0-908ee435260a\") " pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:33.931629 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.931611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34cf3357-ec56-4e44-8b1a-58379395479b-config-volume\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.931660 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.931634 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550040cc-16c3-4d74-80a0-908ee435260a-cert\") pod \"ingress-canary-htk6w\" (UID: \"550040cc-16c3-4d74-80a0-908ee435260a\") " pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:33.931691 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:33.931651 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 18:17:33.931737 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:33.931721 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls podName:34cf3357-ec56-4e44-8b1a-58379395479b nodeName:}" failed. No retries permitted until 2026-07-15 18:17:34.431699747 +0000 UTC m=+35.041512916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls") pod "dns-default-xx5wd" (UID: "34cf3357-ec56-4e44-8b1a-58379395479b") : secret "dns-default-metrics-tls" not found Jul 15 18:17:33.932027 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.932007 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34cf3357-ec56-4e44-8b1a-58379395479b-tmp-dir\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.932160 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.932141 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34cf3357-ec56-4e44-8b1a-58379395479b-config-volume\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.935933 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.935903 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/550040cc-16c3-4d74-80a0-908ee435260a-cert\") pod \"ingress-canary-htk6w\" (UID: \"550040cc-16c3-4d74-80a0-908ee435260a\") " pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:33.942097 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.942077 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqd7j\" (UniqueName: \"kubernetes.io/projected/34cf3357-ec56-4e44-8b1a-58379395479b-kube-api-access-fqd7j\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:33.942097 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:33.942087 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6fpv\" (UniqueName: \"kubernetes.io/projected/550040cc-16c3-4d74-80a0-908ee435260a-kube-api-access-w6fpv\") pod \"ingress-canary-htk6w\" (UID: \"550040cc-16c3-4d74-80a0-908ee435260a\") " pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:34.089279 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.089248 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-htk6w" Jul 15 18:17:34.106267 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.106233 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt"] Jul 15 18:17:34.128991 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.128961 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt"] Jul 15 18:17:34.129082 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.129072 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" Jul 15 18:17:34.131694 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.131657 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:34.131694 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.131666 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cqd94\"" Jul 15 18:17:34.131881 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.131710 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:34.192028 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.191996 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bdcafe2-1ea6-4997-bf00-8471961ef963" containerID="557ad5c9624518009d5c4cc5773a1296e5b8ebb1dd70b2f6020533513eb12229" exitCode=0 Jul 15 18:17:34.192167 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.192047 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerDied","Data":"557ad5c9624518009d5c4cc5773a1296e5b8ebb1dd70b2f6020533513eb12229"} Jul 15 18:17:34.222008 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.221668 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-kplvm"] Jul 15 18:17:34.233622 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.233539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmr8w\" (UniqueName: \"kubernetes.io/projected/a527b261-efe0-492b-9523-6291d637a90d-kube-api-access-tmr8w\") pod \"volume-data-source-validator-78495597f6-gdzmt\" (UID: \"a527b261-efe0-492b-9523-6291d637a90d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" Jul 15 18:17:34.248328 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.248111 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj"] Jul 15 18:17:34.249336 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.249313 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.252720 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.252359 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:34.252720 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.252414 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jk5nr\"" Jul 15 18:17:34.252720 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.252356 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 18:17:34.252720 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.252359 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 18:17:34.253078 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.253052 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:34.259047 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.259023 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 18:17:34.263154 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.262090 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz"] Jul 15 18:17:34.263154 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.262230 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.264832 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.264809 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n297n\"" Jul 15 18:17:34.264953 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.264928 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 18:17:34.265315 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.265296 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 18:17:34.265624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.265605 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:34.265697 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.265606 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:34.284292 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.284269 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm"] Jul 15 18:17:34.284407 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.284340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.286746 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.286725 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 18:17:34.286746 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.286753 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:34.286932 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.286785 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:34.286932 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.286763 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 18:17:34.286932 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.286855 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7gkqz\"" Jul 15 18:17:34.304369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.304349 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kplvm"] Jul 15 18:17:34.304474 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.304378 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj"] Jul 15 18:17:34.304474 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.304391 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz"] Jul 15 18:17:34.304474 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.304468 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:34.304570 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.304485 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm"] Jul 15 18:17:34.304570 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.304508 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-htk6w"] Jul 15 18:17:34.305665 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:34.305640 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550040cc_16c3_4d74_80a0_908ee435260a.slice/crio-401e87bb4d076dc86c3aeeacceb39117ec8fa205dc84b32278dfcab315926b88 WatchSource:0}: Error finding container 401e87bb4d076dc86c3aeeacceb39117ec8fa205dc84b32278dfcab315926b88: Status 404 returned error can't find the container with id 401e87bb4d076dc86c3aeeacceb39117ec8fa205dc84b32278dfcab315926b88 Jul 15 18:17:34.306954 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.306915 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 18:17:34.307532 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.306960 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:34.307639 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.307568 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7bv42\"" Jul 15 18:17:34.307821 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.307800 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 18:17:34.334168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd64h\" (UniqueName: \"kubernetes.io/projected/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-kube-api-access-hd64h\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.334305 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8jd\" (UniqueName: \"kubernetes.io/projected/7f08fedb-4c6a-43eb-86d5-4a56139355a2-kube-api-access-mh8jd\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.334305 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmr8w\" (UniqueName: \"kubernetes.io/projected/a527b261-efe0-492b-9523-6291d637a90d-kube-api-access-tmr8w\") pod \"volume-data-source-validator-78495597f6-gdzmt\" (UID: \"a527b261-efe0-492b-9523-6291d637a90d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a1707097-c6df-4856-b7aa-426736ef40ff-serving-cert\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f08fedb-4c6a-43eb-86d5-4a56139355a2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334382 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334405 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334425 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1707097-c6df-4856-b7aa-426736ef40ff-config\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ddkg\" (UniqueName: \"kubernetes.io/projected/a1707097-c6df-4856-b7aa-426736ef40ff-kube-api-access-8ddkg\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334491 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1707097-c6df-4856-b7aa-426736ef40ff-trusted-ca\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.334514 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.334514 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f08fedb-4c6a-43eb-86d5-4a56139355a2-config\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.343716 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.343643 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmr8w\" (UniqueName: \"kubernetes.io/projected/a527b261-efe0-492b-9523-6291d637a90d-kube-api-access-tmr8w\") pod \"volume-data-source-validator-78495597f6-gdzmt\" (UID: \"a527b261-efe0-492b-9523-6291d637a90d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" Jul 15 18:17:34.435566 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435526 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a1707097-c6df-4856-b7aa-426736ef40ff-serving-cert\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.435566 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435565 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f08fedb-4c6a-43eb-86d5-4a56139355a2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435584 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435651 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1707097-c6df-4856-b7aa-426736ef40ff-config\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435687 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ddkg\" (UniqueName: \"kubernetes.io/projected/a1707097-c6df-4856-b7aa-426736ef40ff-kube-api-access-8ddkg\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435717 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1707097-c6df-4856-b7aa-426736ef40ff-trusted-ca\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435741 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f08fedb-4c6a-43eb-86d5-4a56139355a2-config\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435765 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9sjv\" (UniqueName: \"kubernetes.io/projected/a375aec1-f36a-4e9d-8955-1956df8e5923-kube-api-access-z9sjv\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435819 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd64h\" (UniqueName: \"kubernetes.io/projected/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-kube-api-access-hd64h\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.435839 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:34.435838 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 18:17:34.436352 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.435853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8jd\" (UniqueName: \"kubernetes.io/projected/7f08fedb-4c6a-43eb-86d5-4a56139355a2-kube-api-access-mh8jd\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.436352 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:34.435907 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls podName:34cf3357-ec56-4e44-8b1a-58379395479b nodeName:}" failed. No retries permitted until 2026-07-15 18:17:35.435887158 +0000 UTC m=+36.045700352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls") pod "dns-default-xx5wd" (UID: "34cf3357-ec56-4e44-8b1a-58379395479b") : secret "dns-default-metrics-tls" not found Jul 15 18:17:34.436510 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.436486 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1707097-c6df-4856-b7aa-426736ef40ff-config\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.436599 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.436575 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f08fedb-4c6a-43eb-86d5-4a56139355a2-config\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.436793 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.436752 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.437252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.437230 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1707097-c6df-4856-b7aa-426736ef40ff-trusted-ca\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.438059 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.438033 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f08fedb-4c6a-43eb-86d5-4a56139355a2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.438135 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.438103 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.438135 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.438124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a1707097-c6df-4856-b7aa-426736ef40ff-serving-cert\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.442069 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.442049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" Jul 15 18:17:34.443708 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.443683 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd64h\" (UniqueName: \"kubernetes.io/projected/ea3d3efe-2f9f-4b75-9521-e95e7d1b624a-kube-api-access-hd64h\") pod \"kube-storage-version-migrator-operator-799cb7f958-4pthj\" (UID: \"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.443819 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.443800 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8jd\" (UniqueName: \"kubernetes.io/projected/7f08fedb-4c6a-43eb-86d5-4a56139355a2-kube-api-access-mh8jd\") pod \"service-ca-operator-7695fc6c8c-wrgpz\" (UID: \"7f08fedb-4c6a-43eb-86d5-4a56139355a2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.443876 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.443833 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ddkg\" (UniqueName: \"kubernetes.io/projected/a1707097-c6df-4856-b7aa-426736ef40ff-kube-api-access-8ddkg\") pod \"console-operator-7866497494-kplvm\" (UID: \"a1707097-c6df-4856-b7aa-426736ef40ff\") " pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.536378 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.536330 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9sjv\" (UniqueName: \"kubernetes.io/projected/a375aec1-f36a-4e9d-8955-1956df8e5923-kube-api-access-z9sjv\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:34.536524 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.536461 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:34.536591 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:34.536558 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 18:17:34.536652 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:34.536616 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls podName:a375aec1-f36a-4e9d-8955-1956df8e5923 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:35.036601152 +0000 UTC m=+35.646414322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls") pod "cluster-samples-operator-65db858894-bk6xm" (UID: "a375aec1-f36a-4e9d-8955-1956df8e5923") : secret "samples-operator-tls" not found Jul 15 18:17:34.546474 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.546442 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9sjv\" (UniqueName: \"kubernetes.io/projected/a375aec1-f36a-4e9d-8955-1956df8e5923-kube-api-access-z9sjv\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:34.564141 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.564118 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:34.574252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.574223 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" Jul 15 18:17:34.575871 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.575853 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt"] Jul 15 18:17:34.579883 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:34.579861 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda527b261_efe0_492b_9523_6291d637a90d.slice/crio-eccc0f80cd822b5addda7eca5ed5c7515e9a52723abba9498fd3f4c2b33eea83 WatchSource:0}: Error finding container eccc0f80cd822b5addda7eca5ed5c7515e9a52723abba9498fd3f4c2b33eea83: Status 404 returned error can't find the container with id eccc0f80cd822b5addda7eca5ed5c7515e9a52723abba9498fd3f4c2b33eea83 Jul 15 18:17:34.595793 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.595750 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" Jul 15 18:17:34.728214 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.728048 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kplvm"] Jul 15 18:17:34.731967 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:34.731933 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1707097_c6df_4856_b7aa_426736ef40ff.slice/crio-c6d9c7e473b93b341f1640077ca6092d1d6b34c29c38dedbef18e64a88000428 WatchSource:0}: Error finding container c6d9c7e473b93b341f1640077ca6092d1d6b34c29c38dedbef18e64a88000428: Status 404 returned error can't find the container with id c6d9c7e473b93b341f1640077ca6092d1d6b34c29c38dedbef18e64a88000428 Jul 15 18:17:34.773204 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.768716 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj"] Jul 15 18:17:34.810472 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.810442 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz"] Jul 15 18:17:34.813719 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:34.813689 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f08fedb_4c6a_43eb_86d5_4a56139355a2.slice/crio-3069ffb4f93a3418af22af0c3edfe5be2f603a70d1d728913bb23f452455d62c WatchSource:0}: Error finding container 3069ffb4f93a3418af22af0c3edfe5be2f603a70d1d728913bb23f452455d62c: Status 404 returned error can't find the container with id 3069ffb4f93a3418af22af0c3edfe5be2f603a70d1d728913bb23f452455d62c Jul 15 18:17:34.992419 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.992343 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:17:34.992419 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.992392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:17:34.992608 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.992468 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:17:34.995616 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.995594 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 18:17:34.995741 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.995637 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s7m6p\"" Jul 15 18:17:34.995741 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.995730 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rzhtl\"" Jul 15 18:17:34.995887 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.995597 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 18:17:34.995946 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.995917 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 18:17:34.996072 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:34.996053 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 18:17:35.041629 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.041594 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:35.041889 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:35.041761 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 18:17:35.041889 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:35.041844 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls podName:a375aec1-f36a-4e9d-8955-1956df8e5923 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:36.04182394 +0000 UTC m=+36.651637122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls") pod "cluster-samples-operator-65db858894-bk6xm" (UID: "a375aec1-f36a-4e9d-8955-1956df8e5923") : secret "samples-operator-tls" not found Jul 15 18:17:35.196455 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.196412 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" event={"ID":"a527b261-efe0-492b-9523-6291d637a90d","Type":"ContainerStarted","Data":"eccc0f80cd822b5addda7eca5ed5c7515e9a52723abba9498fd3f4c2b33eea83"} Jul 15 18:17:35.197782 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.197735 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kplvm" event={"ID":"a1707097-c6df-4856-b7aa-426736ef40ff","Type":"ContainerStarted","Data":"c6d9c7e473b93b341f1640077ca6092d1d6b34c29c38dedbef18e64a88000428"} Jul 15 18:17:35.199245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.199216 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-htk6w" event={"ID":"550040cc-16c3-4d74-80a0-908ee435260a","Type":"ContainerStarted","Data":"401e87bb4d076dc86c3aeeacceb39117ec8fa205dc84b32278dfcab315926b88"} Jul 15 18:17:35.200619 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.200569 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" event={"ID":"7f08fedb-4c6a-43eb-86d5-4a56139355a2","Type":"ContainerStarted","Data":"3069ffb4f93a3418af22af0c3edfe5be2f603a70d1d728913bb23f452455d62c"} Jul 15 18:17:35.204837 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.204791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82tdr" event={"ID":"5bdcafe2-1ea6-4997-bf00-8471961ef963","Type":"ContainerStarted","Data":"01609b51c24351f292983751d1c07f530226cc13659e0678822af284e6f2125d"} Jul 15 18:17:35.206082 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.206059 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" event={"ID":"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a","Type":"ContainerStarted","Data":"8b1fbcdc33bda59242dc61b12af48e01f9729771fd0e47ee806b378a643ef43f"} Jul 15 18:17:35.230744 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.230693 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-82tdr" podStartSLOduration=6.6915995200000005 podStartE2EDuration="35.230676305s" totalImagesPullingTime="28.539076785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:35.229677979 +0000 UTC m=+35.839491166" watchObservedRunningTime="2026-07-15 18:17:35.230676305 +0000 UTC m=+35.840489492" Jul 15 18:17:35.444859 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:35.444818 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:35.445085 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:35.445059 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 18:17:35.445203 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:35.445146 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls podName:34cf3357-ec56-4e44-8b1a-58379395479b nodeName:}" failed. No retries permitted until 2026-07-15 18:17:37.44512522 +0000 UTC m=+38.054938388 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls") pod "dns-default-xx5wd" (UID: "34cf3357-ec56-4e44-8b1a-58379395479b") : secret "dns-default-metrics-tls" not found Jul 15 18:17:36.051523 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:36.050948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:36.051523 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:36.051117 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 18:17:36.051523 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:36.051179 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls podName:a375aec1-f36a-4e9d-8955-1956df8e5923 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:38.051160355 +0000 UTC m=+38.660973526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls") pod "cluster-samples-operator-65db858894-bk6xm" (UID: "a375aec1-f36a-4e9d-8955-1956df8e5923") : secret "samples-operator-tls" not found Jul 15 18:17:37.462479 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:37.462442 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:37.462934 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:37.462634 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 18:17:37.462934 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:37.462712 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls podName:34cf3357-ec56-4e44-8b1a-58379395479b nodeName:}" failed. No retries permitted until 2026-07-15 18:17:41.462689785 +0000 UTC m=+42.072502955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls") pod "dns-default-xx5wd" (UID: "34cf3357-ec56-4e44-8b1a-58379395479b") : secret "dns-default-metrics-tls" not found Jul 15 18:17:38.067337 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.067304 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:38.067614 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:38.067443 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 18:17:38.067614 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:38.067508 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls podName:a375aec1-f36a-4e9d-8955-1956df8e5923 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:42.06748899 +0000 UTC m=+42.677302156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls") pod "cluster-samples-operator-65db858894-bk6xm" (UID: "a375aec1-f36a-4e9d-8955-1956df8e5923") : secret "samples-operator-tls" not found Jul 15 18:17:38.546988 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.546954 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qnm4m"] Jul 15 18:17:38.566444 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.566414 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qnm4m"] Jul 15 18:17:38.566619 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.566541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.569131 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.569107 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 18:17:38.569245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.569107 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 18:17:38.570157 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.570135 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bvrpn\"" Jul 15 18:17:38.570261 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.570248 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 18:17:38.570366 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.570351 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 18:17:38.671645 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.671556 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38192496-f9a3-42ff-88db-26a2d9452e2a-data-volume\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.671645 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.671600 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.671881 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.671691 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38192496-f9a3-42ff-88db-26a2d9452e2a-crio-socket\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.671881 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.671709 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.671881 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.671757 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfs7z\" (UniqueName: \"kubernetes.io/projected/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-api-access-nfs7z\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.772762 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.772732 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38192496-f9a3-42ff-88db-26a2d9452e2a-crio-socket\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.772915 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.772779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.772915 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.772818 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfs7z\" (UniqueName: \"kubernetes.io/projected/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-api-access-nfs7z\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.772915 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.772856 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38192496-f9a3-42ff-88db-26a2d9452e2a-data-volume\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.772915 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.772890 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.773124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:38.772926 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 18:17:38.773124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:38.772996 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:39.272977481 +0000 UTC m=+39.882790665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : configmap "kube-rbac-proxy" not found Jul 15 18:17:38.773124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:38.773001 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 18:17:38.773124 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.773001 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38192496-f9a3-42ff-88db-26a2d9452e2a-crio-socket\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.773124 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:38.773037 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:39.273025271 +0000 UTC m=+39.882838438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : secret "insights-runtime-extractor-tls" not found Jul 15 18:17:38.773451 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.773213 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38192496-f9a3-42ff-88db-26a2d9452e2a-data-volume\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:38.784541 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:38.784520 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfs7z\" (UniqueName: \"kubernetes.io/projected/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-api-access-nfs7z\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:39.219185 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.219095 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" event={"ID":"7f08fedb-4c6a-43eb-86d5-4a56139355a2","Type":"ContainerStarted","Data":"54bdb9583530d878d30dfd1b7d0277f79b1bca9c03ba018eee418def519b6c8d"} Jul 15 18:17:39.220663 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.220622 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" event={"ID":"a527b261-efe0-492b-9523-6291d637a90d","Type":"ContainerStarted","Data":"fb2f53b398e11d16d84f7b7ebfec89fc7e6c7b1c0b8c3e8a160c4c7a18f93314"} Jul 15 18:17:39.222326 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.222302 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/0.log" Jul 15 18:17:39.222442 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.222339 2557 generic.go:356] "Generic (PLEG): container finished" podID="a1707097-c6df-4856-b7aa-426736ef40ff" containerID="f94c579774646af7ef84b5beac13a018f3ac00287c1f4798b292d3602a4cdc0e" exitCode=255 Jul 15 18:17:39.222442 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.222416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kplvm" event={"ID":"a1707097-c6df-4856-b7aa-426736ef40ff","Type":"ContainerDied","Data":"f94c579774646af7ef84b5beac13a018f3ac00287c1f4798b292d3602a4cdc0e"} Jul 15 18:17:39.222657 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.222639 2557 scope.go:117] "RemoveContainer" containerID="f94c579774646af7ef84b5beac13a018f3ac00287c1f4798b292d3602a4cdc0e" Jul 15 18:17:39.223930 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.223908 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-htk6w" event={"ID":"550040cc-16c3-4d74-80a0-908ee435260a","Type":"ContainerStarted","Data":"7b2c5304bf0d70b09c4edfa2711d8e87aef83458b29eb30da99cffbfbe0235cc"} Jul 15 18:17:39.236624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.236583 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" podStartSLOduration=1.638200562 podStartE2EDuration="5.236569483s" totalImagesPullingTime="3.598368921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:39.235959687 +0000 UTC m=+39.845772876" watchObservedRunningTime="2026-07-15 18:17:39.236569483 +0000 UTC m=+39.846382670" Jul 15 18:17:39.267166 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.267118 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-htk6w" podStartSLOduration=2.169911061 podStartE2EDuration="6.267101945s" totalImagesPullingTime="4.097190884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:39.266521731 +0000 UTC m=+39.876334923" watchObservedRunningTime="2026-07-15 18:17:39.267101945 +0000 UTC m=+39.876915173" Jul 15 18:17:39.277826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.277794 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:39.278001 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:39.277959 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 18:17:39.278065 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:39.278021 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:40.2780017 +0000 UTC m=+40.887814865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : secret "insights-runtime-extractor-tls" not found Jul 15 18:17:39.278124 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.278095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:39.278657 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:39.278559 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 18:17:39.278657 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:39.278613 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:40.278596585 +0000 UTC m=+40.888409751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : configmap "kube-rbac-proxy" not found Jul 15 18:17:39.280981 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:39.280926 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gdzmt" podStartSLOduration=1.511889072 podStartE2EDuration="5.280913761s" totalImagesPullingTime="3.769024689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:39.280868365 +0000 UTC m=+39.890681553" watchObservedRunningTime="2026-07-15 18:17:39.280913761 +0000 UTC m=+39.890726948" Jul 15 18:17:40.038337 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.038306 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-56pv4_f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e/dns-node-resolver/0.log" Jul 15 18:17:40.228705 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.228667 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:17:40.229066 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.229052 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/0.log" Jul 15 18:17:40.229151 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.229082 2557 generic.go:356] "Generic (PLEG): container finished" podID="a1707097-c6df-4856-b7aa-426736ef40ff" containerID="88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e" exitCode=255 Jul 15 18:17:40.229208 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.229161 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kplvm" event={"ID":"a1707097-c6df-4856-b7aa-426736ef40ff","Type":"ContainerDied","Data":"88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e"} Jul 15 18:17:40.229265 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.229220 2557 scope.go:117] "RemoveContainer" containerID="f94c579774646af7ef84b5beac13a018f3ac00287c1f4798b292d3602a4cdc0e" Jul 15 18:17:40.229524 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.229503 2557 scope.go:117] "RemoveContainer" containerID="88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e" Jul 15 18:17:40.229724 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:40.229698 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kplvm_openshift-console-operator(a1707097-c6df-4856-b7aa-426736ef40ff)\"" pod="openshift-console-operator/console-operator-7866497494-kplvm" podUID="a1707097-c6df-4856-b7aa-426736ef40ff" Jul 15 18:17:40.230788 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.230751 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" event={"ID":"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a","Type":"ContainerStarted","Data":"e889dbdf9b7d67e9993312d7aaefd97d6b5a0a9ffa900bdf953cfa7f642bd7c8"} Jul 15 18:17:40.285972 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.285937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:40.286664 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:40.286509 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:40.287211 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:40.286442 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 18:17:40.287211 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:40.286880 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:42.286856339 +0000 UTC m=+42.896669509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : configmap "kube-rbac-proxy" not found Jul 15 18:17:40.287211 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:40.286634 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 18:17:40.287211 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:40.286988 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:42.286966939 +0000 UTC m=+42.896780108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : secret "insights-runtime-extractor-tls" not found Jul 15 18:17:41.236270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.236242 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:17:41.236671 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.236598 2557 scope.go:117] "RemoveContainer" containerID="88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e" Jul 15 18:17:41.236829 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:41.236804 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kplvm_openshift-console-operator(a1707097-c6df-4856-b7aa-426736ef40ff)\"" pod="openshift-console-operator/console-operator-7866497494-kplvm" podUID="a1707097-c6df-4856-b7aa-426736ef40ff" Jul 15 18:17:41.238476 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.238458 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd77m_19430ccd-cec5-413d-a155-58a1049530be/node-ca/0.log" Jul 15 18:17:41.252880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.252840 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" podStartSLOduration=2.643883034 podStartE2EDuration="7.252827561s" totalImagesPullingTime="4.608944527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:40.263374358 +0000 UTC m=+40.873187545" watchObservedRunningTime="2026-07-15 18:17:41.252827561 +0000 UTC m=+41.862640796" Jul 15 18:17:41.497761 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.497672 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:41.498002 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:41.497842 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 18:17:41.498002 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:41.497922 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls podName:34cf3357-ec56-4e44-8b1a-58379395479b nodeName:}" failed. No retries permitted until 2026-07-15 18:17:49.49790135 +0000 UTC m=+50.107714516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls") pod "dns-default-xx5wd" (UID: "34cf3357-ec56-4e44-8b1a-58379395479b") : secret "dns-default-metrics-tls" not found Jul 15 18:17:41.645986 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.645953 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-htk6w_550040cc-16c3-4d74-80a0-908ee435260a/serve-healthcheck-canary/0.log" Jul 15 18:17:41.668673 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.668640 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb"] Jul 15 18:17:41.701979 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.701943 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb"] Jul 15 18:17:41.702120 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.702056 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" Jul 15 18:17:41.704488 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.704464 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 18:17:41.704698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.704682 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 18:17:41.704828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.704809 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-rv6lm\"" Jul 15 18:17:41.800227 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.800138 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmj62\" (UniqueName: \"kubernetes.io/projected/ba8f959a-ecc5-466b-b411-2198a71967cd-kube-api-access-kmj62\") pod \"migrator-7f59b54f6c-mzhxb\" (UID: \"ba8f959a-ecc5-466b-b411-2198a71967cd\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" Jul 15 18:17:41.901069 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.901030 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmj62\" (UniqueName: \"kubernetes.io/projected/ba8f959a-ecc5-466b-b411-2198a71967cd-kube-api-access-kmj62\") pod \"migrator-7f59b54f6c-mzhxb\" (UID: \"ba8f959a-ecc5-466b-b411-2198a71967cd\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" Jul 15 18:17:41.909703 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:41.909670 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmj62\" (UniqueName: \"kubernetes.io/projected/ba8f959a-ecc5-466b-b411-2198a71967cd-kube-api-access-kmj62\") pod \"migrator-7f59b54f6c-mzhxb\" (UID: \"ba8f959a-ecc5-466b-b411-2198a71967cd\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" Jul 15 18:17:42.011493 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:42.011465 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" Jul 15 18:17:42.102425 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:42.102385 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:42.102606 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:42.102538 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 18:17:42.102667 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:42.102638 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls podName:a375aec1-f36a-4e9d-8955-1956df8e5923 nodeName:}" failed. No retries permitted until 2026-07-15 18:17:50.102617231 +0000 UTC m=+50.712430398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls") pod "cluster-samples-operator-65db858894-bk6xm" (UID: "a375aec1-f36a-4e9d-8955-1956df8e5923") : secret "samples-operator-tls" not found Jul 15 18:17:42.153214 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:42.153187 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb"] Jul 15 18:17:42.156109 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:42.156081 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8f959a_ecc5_466b_b411_2198a71967cd.slice/crio-5388d557d2022c849268f3f1014fb0a07b006d09d93ec4cb682429fc17acd621 WatchSource:0}: Error finding container 5388d557d2022c849268f3f1014fb0a07b006d09d93ec4cb682429fc17acd621: Status 404 returned error can't find the container with id 5388d557d2022c849268f3f1014fb0a07b006d09d93ec4cb682429fc17acd621 Jul 15 18:17:42.240379 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:42.240344 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" event={"ID":"ba8f959a-ecc5-466b-b411-2198a71967cd","Type":"ContainerStarted","Data":"5388d557d2022c849268f3f1014fb0a07b006d09d93ec4cb682429fc17acd621"} Jul 15 18:17:42.304050 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:42.304010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:42.304301 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:42.304118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:42.304301 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:42.304178 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 18:17:42.304301 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:42.304218 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 18:17:42.304301 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:42.304242 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:46.304226886 +0000 UTC m=+46.914040052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : secret "insights-runtime-extractor-tls" not found Jul 15 18:17:42.304301 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:42.304268 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:46.304253681 +0000 UTC m=+46.914066853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : configmap "kube-rbac-proxy" not found Jul 15 18:17:44.247732 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:44.247651 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" event={"ID":"ba8f959a-ecc5-466b-b411-2198a71967cd","Type":"ContainerStarted","Data":"9748e5ea527c5198c187ed5411e05ef5ea3c2c2fa5dd56eca005509e6feb31cc"} Jul 15 18:17:44.247732 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:44.247701 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" event={"ID":"ba8f959a-ecc5-466b-b411-2198a71967cd","Type":"ContainerStarted","Data":"686c5823589a26b8fa4a827f9e6bb6e42c656860aa99a8bdaa1880780c1202a6"} Jul 15 18:17:44.265305 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:44.265249 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mzhxb" podStartSLOduration=1.45970173 podStartE2EDuration="3.265232491s" totalImagesPullingTime="1.805530761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:44.263817614 +0000 UTC m=+44.873630802" watchObservedRunningTime="2026-07-15 18:17:44.265232491 +0000 UTC m=+44.875045679" Jul 15 18:17:44.565017 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:44.564931 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:44.565017 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:44.564996 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:44.565315 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:44.565300 2557 scope.go:117] "RemoveContainer" containerID="88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e" Jul 15 18:17:44.565461 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:44.565444 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kplvm_openshift-console-operator(a1707097-c6df-4856-b7aa-426736ef40ff)\"" pod="openshift-console-operator/console-operator-7866497494-kplvm" podUID="a1707097-c6df-4856-b7aa-426736ef40ff" Jul 15 18:17:45.250975 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:45.250949 2557 scope.go:117] "RemoveContainer" containerID="88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e" Jul 15 18:17:45.251352 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:45.251125 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kplvm_openshift-console-operator(a1707097-c6df-4856-b7aa-426736ef40ff)\"" pod="openshift-console-operator/console-operator-7866497494-kplvm" podUID="a1707097-c6df-4856-b7aa-426736ef40ff" Jul 15 18:17:46.338928 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:46.338884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:46.339312 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:46.338993 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:46.339312 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:46.339051 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 18:17:46.339312 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:46.339112 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 18:17:46.339312 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:46.339120 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:54.33910552 +0000 UTC m=+54.948918688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : secret "insights-runtime-extractor-tls" not found Jul 15 18:17:46.339312 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:46.339173 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm podName:38192496-f9a3-42ff-88db-26a2d9452e2a nodeName:}" failed. No retries permitted until 2026-07-15 18:17:54.33915984 +0000 UTC m=+54.948973006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnm4m" (UID: "38192496-f9a3-42ff-88db-26a2d9452e2a") : configmap "kube-rbac-proxy" not found Jul 15 18:17:49.565301 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:49.565263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:17:49.565926 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:49.565422 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 18:17:49.565926 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:17:49.565500 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls podName:34cf3357-ec56-4e44-8b1a-58379395479b nodeName:}" failed. No retries permitted until 2026-07-15 18:18:05.565483399 +0000 UTC m=+66.175296565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls") pod "dns-default-xx5wd" (UID: "34cf3357-ec56-4e44-8b1a-58379395479b") : secret "dns-default-metrics-tls" not found Jul 15 18:17:50.170008 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:50.169966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:50.172312 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:50.172289 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a375aec1-f36a-4e9d-8955-1956df8e5923-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-bk6xm\" (UID: \"a375aec1-f36a-4e9d-8955-1956df8e5923\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:50.216844 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:50.216803 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" Jul 15 18:17:50.353917 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:50.353880 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm"] Jul 15 18:17:51.275062 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:51.275029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" event={"ID":"a375aec1-f36a-4e9d-8955-1956df8e5923","Type":"ContainerStarted","Data":"40ea450f1a1747851ce9f66a4757918a3f059f93c0f532faca4874e6a3b4a612"} Jul 15 18:17:53.282908 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:53.282867 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" event={"ID":"a375aec1-f36a-4e9d-8955-1956df8e5923","Type":"ContainerStarted","Data":"b7e278474127272a83b4a704322e31f18713ef8c73c09f6da2e186c7b4e3e172"} Jul 15 18:17:53.283378 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:53.282926 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" event={"ID":"a375aec1-f36a-4e9d-8955-1956df8e5923","Type":"ContainerStarted","Data":"b05da8ba4046528986229c56eae7c8ba50bdb288c077eb730d830f9b73849012"} Jul 15 18:17:53.299617 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:53.299574 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-bk6xm" podStartSLOduration=17.198192368 podStartE2EDuration="19.299559106s" totalImagesPullingTime="2.101366738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:53.299338938 +0000 UTC m=+53.909152126" watchObservedRunningTime="2026-07-15 18:17:53.299559106 +0000 UTC m=+53.909372293" Jul 15 18:17:54.405395 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:54.405356 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:54.405795 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:54.405468 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:54.406063 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:54.406039 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38192496-f9a3-42ff-88db-26a2d9452e2a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:54.407698 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:54.407677 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38192496-f9a3-42ff-88db-26a2d9452e2a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnm4m\" (UID: \"38192496-f9a3-42ff-88db-26a2d9452e2a\") " pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:54.477607 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:54.477574 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qnm4m" Jul 15 18:17:54.596700 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:54.596523 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qnm4m"] Jul 15 18:17:54.599095 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:17:54.599069 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38192496_f9a3_42ff_88db_26a2d9452e2a.slice/crio-a3eb5fa09dd64c0c843c39f68a77dd3fae8598f62cb6ae2583dbdf299c438221 WatchSource:0}: Error finding container a3eb5fa09dd64c0c843c39f68a77dd3fae8598f62cb6ae2583dbdf299c438221: Status 404 returned error can't find the container with id a3eb5fa09dd64c0c843c39f68a77dd3fae8598f62cb6ae2583dbdf299c438221 Jul 15 18:17:55.290488 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:55.290457 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnm4m" event={"ID":"38192496-f9a3-42ff-88db-26a2d9452e2a","Type":"ContainerStarted","Data":"a3eb5fa09dd64c0c843c39f68a77dd3fae8598f62cb6ae2583dbdf299c438221"} Jul 15 18:17:56.992464 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:56.992433 2557 scope.go:117] "RemoveContainer" containerID="88b3ef0d108262f573056a68aacc3ff0562df5a518b40094b6982138917fde1e" Jul 15 18:17:57.300460 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:57.300383 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:17:57.300460 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:57.300450 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kplvm" event={"ID":"a1707097-c6df-4856-b7aa-426736ef40ff","Type":"ContainerStarted","Data":"8c1a3a70a4d141bdee42b9990323b44fb5d53b9dd580f8e6a35d9c75c2d11b93"} Jul 15 18:17:57.300951 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:57.300925 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:17:57.321746 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:57.321695 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-kplvm" podStartSLOduration=19.646869053 podStartE2EDuration="23.321680083s" totalImagesPullingTime="3.67481103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:17:57.320882318 +0000 UTC m=+57.930695505" watchObservedRunningTime="2026-07-15 18:17:57.321680083 +0000 UTC m=+57.931493271" Jul 15 18:17:58.225616 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:17:58.225580 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-kplvm" Jul 15 18:18:01.194873 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:01.193821 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlr77" Jul 15 18:18:01.317602 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:01.317564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnm4m" event={"ID":"38192496-f9a3-42ff-88db-26a2d9452e2a","Type":"ContainerStarted","Data":"eedcee29b2c3b0f26bbeef702df8f4f94e2d987a42d90ff520f78388d1861f52"} Jul 15 18:18:03.346589 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.346417 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tzkcr"] Jul 15 18:18:03.408609 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.408567 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx"] Jul 15 18:18:03.408807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.408691 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.413331 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.413295 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-77cn6\"" Jul 15 18:18:03.413611 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.413594 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 18:18:03.415305 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.413806 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 18:18:03.429686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.429656 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt"] Jul 15 18:18:03.429862 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.429844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.432428 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.432400 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 18:18:03.432544 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.432405 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qbkfs\"" Jul 15 18:18:03.432544 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.432506 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 18:18:03.433229 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.433210 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 18:18:03.433447 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.433431 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 15 18:18:03.458138 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.458112 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tzkcr"] Jul 15 18:18:03.458312 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.458152 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx"] Jul 15 18:18:03.458312 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.458166 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt"] Jul 15 18:18:03.458312 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.458190 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-658c648df6-bnd2z"] Jul 15 18:18:03.458312 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.458250 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.461082 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.461047 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 15 18:18:03.461254 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.461092 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 15 18:18:03.461254 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.461108 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 15 18:18:03.461254 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.461176 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 15 18:18:03.477520 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.477483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66959d9bd8-q2lwx\" (UID: \"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.477691 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.477539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pclrc\" (UniqueName: \"kubernetes.io/projected/8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb-kube-api-access-pclrc\") pod \"managed-serviceaccount-addon-agent-66959d9bd8-q2lwx\" (UID: \"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.486083 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.486056 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658c648df6-bnd2z"] Jul 15 18:18:03.486718 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.486178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.488372 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.488316 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-658c648df6-bnd2z"] Jul 15 18:18:03.488817 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.488757 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 18:18:03.488925 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.488878 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 18:18:03.488925 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.488915 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cknmp\"" Jul 15 18:18:03.490125 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.490098 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 18:18:03.493643 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.493626 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 18:18:03.509374 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:18:03.509345 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-bgdv7 registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-bgdv7 registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-658c648df6-bnd2z" podUID="d027af08-cc86-448a-8251-d186c2209582" Jul 15 18:18:03.578525 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-hub\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.578686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578533 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.578686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-registry-certificates\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.578686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578587 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-registry-tls\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.578686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578601 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-bound-sa-token\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.578686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578628 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tzkcr\" (UID: \"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.578936 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578686 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bs75\" (UniqueName: \"kubernetes.io/projected/320bfd49-dfcb-45c4-9589-aa82688b0140-kube-api-access-7bs75\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.578936 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578727 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d027af08-cc86-448a-8251-d186c2209582-ca-trust-extracted\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.578936 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578777 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tzkcr\" (UID: \"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.578936 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578848 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-ca\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.578936 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgdv7\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-kube-api-access-bgdv7\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.578936 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578904 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-installation-pull-secrets\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.579210 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.578963 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.579210 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.579006 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66959d9bd8-q2lwx\" (UID: \"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.579210 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.579038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-image-registry-private-configuration\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.579210 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.579071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pclrc\" (UniqueName: \"kubernetes.io/projected/8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb-kube-api-access-pclrc\") pod \"managed-serviceaccount-addon-agent-66959d9bd8-q2lwx\" (UID: \"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.579210 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.579102 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-trusted-ca\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.579478 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.579269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/320bfd49-dfcb-45c4-9589-aa82688b0140-additional-service-ca\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.581953 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.581923 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66959d9bd8-q2lwx\" (UID: \"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.588447 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.588422 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pclrc\" (UniqueName: \"kubernetes.io/projected/8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb-kube-api-access-pclrc\") pod \"managed-serviceaccount-addon-agent-66959d9bd8-q2lwx\" (UID: \"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.680243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680165 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.680243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-registry-certificates\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.680243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-registry-tls\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.680243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680244 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-bound-sa-token\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tzkcr\" (UID: \"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bs75\" (UniqueName: \"kubernetes.io/projected/320bfd49-dfcb-45c4-9589-aa82688b0140-kube-api-access-7bs75\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d027af08-cc86-448a-8251-d186c2209582-ca-trust-extracted\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680371 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tzkcr\" (UID: \"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-ca\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680432 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgdv7\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-kube-api-access-bgdv7\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680463 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-installation-pull-secrets\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.680547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680525 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-image-registry-private-configuration\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.681030 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-trusted-ca\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.681030 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680589 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/320bfd49-dfcb-45c4-9589-aa82688b0140-additional-service-ca\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.681030 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.680658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-hub\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.681180 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.681145 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-registry-certificates\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.681418 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.681393 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d027af08-cc86-448a-8251-d186c2209582-ca-trust-extracted\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.681501 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.681456 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9-nginx-conf\") pod \"networking-console-plugin-6cf856758c-tzkcr\" (UID: \"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.682129 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.682106 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-trusted-ca\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.682243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.682134 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/320bfd49-dfcb-45c4-9589-aa82688b0140-additional-service-ca\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.683510 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.683464 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-hub\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.683983 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.683864 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-ca\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.684081 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.684062 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.684273 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.684227 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-installation-pull-secrets\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.684574 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.684545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-registry-tls\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.684718 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.684693 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-tzkcr\" (UID: \"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.685473 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.685453 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-image-registry-private-configuration\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.686506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.686490 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/320bfd49-dfcb-45c4-9589-aa82688b0140-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.692558 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.692536 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-bound-sa-token\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.692905 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.692685 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgdv7\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-kube-api-access-bgdv7\") pod \"image-registry-658c648df6-bnd2z\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:03.692905 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.692889 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bs75\" (UniqueName: \"kubernetes.io/projected/320bfd49-dfcb-45c4-9589-aa82688b0140-kube-api-access-7bs75\") pod \"cluster-proxy-proxy-agent-676f6ddb-x2qqt\" (UID: \"320bfd49-dfcb-45c4-9589-aa82688b0140\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:03.724171 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.724142 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" Jul 15 18:18:03.749468 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.749439 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" Jul 15 18:18:03.772230 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:03.772199 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" Jul 15 18:18:04.123687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.123658 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt"] Jul 15 18:18:04.127099 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:04.127076 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod320bfd49_dfcb_45c4_9589_aa82688b0140.slice/crio-78ceba363856b6e3be8fa410b3e3f444bb3be85bcaebe9488e50c52a5a2592d3 WatchSource:0}: Error finding container 78ceba363856b6e3be8fa410b3e3f444bb3be85bcaebe9488e50c52a5a2592d3: Status 404 returned error can't find the container with id 78ceba363856b6e3be8fa410b3e3f444bb3be85bcaebe9488e50c52a5a2592d3 Jul 15 18:18:04.329968 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.329929 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" event={"ID":"320bfd49-dfcb-45c4-9589-aa82688b0140","Type":"ContainerStarted","Data":"78ceba363856b6e3be8fa410b3e3f444bb3be85bcaebe9488e50c52a5a2592d3"} Jul 15 18:18:04.330115 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.329942 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:04.342094 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.342066 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx"] Jul 15 18:18:04.342992 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.342958 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-tzkcr"] Jul 15 18:18:04.345004 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:04.344984 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod305b6e0c_0c28_4d1e_bacf_09ae4e2af7d9.slice/crio-65d367015d38f2320c6a356ad2d3f72f385afc3321904dbeab61e7ad8b1fef48 WatchSource:0}: Error finding container 65d367015d38f2320c6a356ad2d3f72f385afc3321904dbeab61e7ad8b1fef48: Status 404 returned error can't find the container with id 65d367015d38f2320c6a356ad2d3f72f385afc3321904dbeab61e7ad8b1fef48 Jul 15 18:18:04.345686 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:04.345615 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dfdf9af_ead0_4cdd_bc39_6a65e5b40aeb.slice/crio-92d7462fd7501835ffce7690df5cd3d36d24d618a231b773da922299a7e73141 WatchSource:0}: Error finding container 92d7462fd7501835ffce7690df5cd3d36d24d618a231b773da922299a7e73141: Status 404 returned error can't find the container with id 92d7462fd7501835ffce7690df5cd3d36d24d618a231b773da922299a7e73141 Jul 15 18:18:04.353690 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.353671 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:04.386680 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386655 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-trusted-ca\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386813 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386708 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-registry-tls\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386857 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386828 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-bound-sa-token\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386897 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386861 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d027af08-cc86-448a-8251-d186c2209582-ca-trust-extracted\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386897 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386883 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgdv7\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-kube-api-access-bgdv7\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386996 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386907 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-installation-pull-secrets\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386996 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386941 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-registry-certificates\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.386996 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.386972 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-image-registry-private-configuration\") pod \"d027af08-cc86-448a-8251-d186c2209582\" (UID: \"d027af08-cc86-448a-8251-d186c2209582\") " Jul 15 18:18:04.387217 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.387188 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d027af08-cc86-448a-8251-d186c2209582-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:18:04.387217 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.387203 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:18:04.387599 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.387538 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:18:04.388979 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.388955 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:18:04.389058 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.389001 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:18:04.389120 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.389102 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-kube-api-access-bgdv7" (OuterVolumeSpecName: "kube-api-access-bgdv7") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "kube-api-access-bgdv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:18:04.389413 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.389389 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 18:18:04.389413 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.389397 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d027af08-cc86-448a-8251-d186c2209582" (UID: "d027af08-cc86-448a-8251-d186c2209582"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:18:04.488101 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488073 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-registry-tls\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488101 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488097 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-bound-sa-token\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488101 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488106 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d027af08-cc86-448a-8251-d186c2209582-ca-trust-extracted\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488321 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488115 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgdv7\" (UniqueName: \"kubernetes.io/projected/d027af08-cc86-448a-8251-d186c2209582-kube-api-access-bgdv7\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488321 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488124 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-installation-pull-secrets\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488321 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488132 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-registry-certificates\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488321 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488140 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d027af08-cc86-448a-8251-d186c2209582-image-registry-private-configuration\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.488321 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.488149 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d027af08-cc86-448a-8251-d186c2209582-trusted-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:18:04.790137 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.790086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:18:04.790308 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.790233 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:18:04.794679 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.794494 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 18:18:04.794679 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.794586 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 18:18:04.803843 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.803816 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97944bdc-1dea-4012-ac18-4827adcd2da6-metrics-certs\") pod \"network-metrics-daemon-csskv\" (UID: \"97944bdc-1dea-4012-ac18-4827adcd2da6\") " pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:18:04.803973 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.803913 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19672c70-677e-46a2-9a56-399fc656d1a5-original-pull-secret\") pod \"global-pull-secret-syncer-2n6fz\" (UID: \"19672c70-677e-46a2-9a56-399fc656d1a5\") " pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:18:04.891642 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.891538 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:18:04.893990 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.893964 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 18:18:04.904340 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.904152 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 18:18:04.915432 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:04.915409 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cm7l\" (UniqueName: \"kubernetes.io/projected/6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1-kube-api-access-9cm7l\") pod \"network-check-target-h9rmb\" (UID: \"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1\") " pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:18:05.004401 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.004364 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2n6fz" Jul 15 18:18:05.013357 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.013149 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rzhtl\"" Jul 15 18:18:05.019784 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.019490 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s7m6p\"" Jul 15 18:18:05.023476 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.023453 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csskv" Jul 15 18:18:05.028073 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.027903 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:18:05.219428 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.217120 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2n6fz"] Jul 15 18:18:05.226787 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:05.226730 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19672c70_677e_46a2_9a56_399fc656d1a5.slice/crio-649a7754481f36b475227377d54f35a49ee629a6b4201fa533bccdfabdbcc4c1 WatchSource:0}: Error finding container 649a7754481f36b475227377d54f35a49ee629a6b4201fa533bccdfabdbcc4c1: Status 404 returned error can't find the container with id 649a7754481f36b475227377d54f35a49ee629a6b4201fa533bccdfabdbcc4c1 Jul 15 18:18:05.254841 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.254709 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-csskv"] Jul 15 18:18:05.261097 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:05.261037 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97944bdc_1dea_4012_ac18_4827adcd2da6.slice/crio-a5adad981da56a1a6c7df870a3c9d412d1af0ecf0544a95afa6cae3641e9ac2b WatchSource:0}: Error finding container a5adad981da56a1a6c7df870a3c9d412d1af0ecf0544a95afa6cae3641e9ac2b: Status 404 returned error can't find the container with id a5adad981da56a1a6c7df870a3c9d412d1af0ecf0544a95afa6cae3641e9ac2b Jul 15 18:18:05.278319 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.278280 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h9rmb"] Jul 15 18:18:05.282964 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:05.282934 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aaeba4e_5969_4d6a_992d_4ddfaf2dedb1.slice/crio-4a74e943ef343bcef3cba1e5eb3abe241e0401aa8f5279cdb03e6f5d9ef8b04c WatchSource:0}: Error finding container 4a74e943ef343bcef3cba1e5eb3abe241e0401aa8f5279cdb03e6f5d9ef8b04c: Status 404 returned error can't find the container with id 4a74e943ef343bcef3cba1e5eb3abe241e0401aa8f5279cdb03e6f5d9ef8b04c Jul 15 18:18:05.338994 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.338884 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnm4m" event={"ID":"38192496-f9a3-42ff-88db-26a2d9452e2a","Type":"ContainerStarted","Data":"f6cea156016e1f127eae1ff417deb6ace28fa9c2a0b10bc140f0c9343d27a3d0"} Jul 15 18:18:05.340889 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.340816 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" event={"ID":"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb","Type":"ContainerStarted","Data":"92d7462fd7501835ffce7690df5cd3d36d24d618a231b773da922299a7e73141"} Jul 15 18:18:05.343661 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.343580 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h9rmb" event={"ID":"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1","Type":"ContainerStarted","Data":"4a74e943ef343bcef3cba1e5eb3abe241e0401aa8f5279cdb03e6f5d9ef8b04c"} Jul 15 18:18:05.345723 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.345657 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2n6fz" event={"ID":"19672c70-677e-46a2-9a56-399fc656d1a5","Type":"ContainerStarted","Data":"649a7754481f36b475227377d54f35a49ee629a6b4201fa533bccdfabdbcc4c1"} Jul 15 18:18:05.347406 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.347335 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" event={"ID":"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9","Type":"ContainerStarted","Data":"65d367015d38f2320c6a356ad2d3f72f385afc3321904dbeab61e7ad8b1fef48"} Jul 15 18:18:05.355087 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.349760 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658c648df6-bnd2z" Jul 15 18:18:05.355087 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.354897 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-csskv" event={"ID":"97944bdc-1dea-4012-ac18-4827adcd2da6","Type":"ContainerStarted","Data":"a5adad981da56a1a6c7df870a3c9d412d1af0ecf0544a95afa6cae3641e9ac2b"} Jul 15 18:18:05.391796 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.391640 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64c86bbcb8-d6ggr"] Jul 15 18:18:05.396539 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.396514 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-658c648df6-bnd2z"] Jul 15 18:18:05.396690 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.396638 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.400245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.400173 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-658c648df6-bnd2z"] Jul 15 18:18:05.401660 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.401639 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 18:18:05.403409 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.401642 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 18:18:05.403409 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.401954 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 18:18:05.403409 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.402016 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cknmp\"" Jul 15 18:18:05.409066 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.408048 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c86bbcb8-d6ggr"] Jul 15 18:18:05.410809 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.410762 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 18:18:05.497044 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.496952 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-ca-trust-extracted\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497044 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497005 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-registry-certificates\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497044 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497037 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-trusted-ca\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497508 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-bound-sa-token\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497508 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-image-registry-private-configuration\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497508 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-registry-tls\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497508 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497215 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-installation-pull-secrets\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.497508 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.497246 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmphj\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-kube-api-access-kmphj\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597735 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-installation-pull-secrets\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmphj\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-kube-api-access-kmphj\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597874 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-ca-trust-extracted\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-registry-certificates\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597942 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-trusted-ca\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.597966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-bound-sa-token\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.598007 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-image-registry-private-configuration\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.598376 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.598039 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-registry-tls\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.601310 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.599071 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-registry-certificates\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.601310 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.599364 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-ca-trust-extracted\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.601310 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.600886 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-trusted-ca\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.624858 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.624799 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmphj\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-kube-api-access-kmphj\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.625275 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.625226 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-bound-sa-token\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.626897 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.626838 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34cf3357-ec56-4e44-8b1a-58379395479b-metrics-tls\") pod \"dns-default-xx5wd\" (UID: \"34cf3357-ec56-4e44-8b1a-58379395479b\") " pod="openshift-dns/dns-default-xx5wd" Jul 15 18:18:05.627511 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.627464 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-registry-tls\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.628098 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.628048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-installation-pull-secrets\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.631118 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.631072 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c8bba59a-4efb-41f8-817b-8a04c1bee4b6-image-registry-private-configuration\") pod \"image-registry-64c86bbcb8-d6ggr\" (UID: \"c8bba59a-4efb-41f8-817b-8a04c1bee4b6\") " pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.715663 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.715626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:05.873790 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.873749 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-42x79\"" Jul 15 18:18:05.882243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.881941 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx5wd" Jul 15 18:18:05.918901 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:05.918821 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c86bbcb8-d6ggr"] Jul 15 18:18:06.001090 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:06.000878 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d027af08-cc86-448a-8251-d186c2209582" path="/var/lib/kubelet/pods/d027af08-cc86-448a-8251-d186c2209582/volumes" Jul 15 18:18:06.507724 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:06.507682 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8bba59a_4efb_41f8_817b_8a04c1bee4b6.slice/crio-f8301d134b6244ccdefc277f1c6c5882f35e84ce38542b693a475ace95b4e189 WatchSource:0}: Error finding container f8301d134b6244ccdefc277f1c6c5882f35e84ce38542b693a475ace95b4e189: Status 404 returned error can't find the container with id f8301d134b6244ccdefc277f1c6c5882f35e84ce38542b693a475ace95b4e189 Jul 15 18:18:07.363150 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:07.363096 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" event={"ID":"c8bba59a-4efb-41f8-817b-8a04c1bee4b6","Type":"ContainerStarted","Data":"f8301d134b6244ccdefc277f1c6c5882f35e84ce38542b693a475ace95b4e189"} Jul 15 18:18:10.000534 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:10.000508 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xx5wd"] Jul 15 18:18:11.706956 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:11.706914 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34cf3357_ec56_4e44_8b1a_58379395479b.slice/crio-0a95c93635eacff479223ffe00fe2bedda4fdb4d2efc6e6321fa21b7b01b78d5 WatchSource:0}: Error finding container 0a95c93635eacff479223ffe00fe2bedda4fdb4d2efc6e6321fa21b7b01b78d5: Status 404 returned error can't find the container with id 0a95c93635eacff479223ffe00fe2bedda4fdb4d2efc6e6321fa21b7b01b78d5 Jul 15 18:18:12.384839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:12.384802 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx5wd" event={"ID":"34cf3357-ec56-4e44-8b1a-58379395479b","Type":"ContainerStarted","Data":"0a95c93635eacff479223ffe00fe2bedda4fdb4d2efc6e6321fa21b7b01b78d5"} Jul 15 18:18:15.427097 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:15.427068 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" event={"ID":"c8bba59a-4efb-41f8-817b-8a04c1bee4b6","Type":"ContainerStarted","Data":"874bb711edd1401fe662ff5b0c9e49d8c4f5006d286b5c98b1db40305de3a683"} Jul 15 18:18:15.429625 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:15.428080 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:15.449438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:15.449388 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" podStartSLOduration=12.449367 podStartE2EDuration="12.449367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:15.448639805 +0000 UTC m=+76.058453007" watchObservedRunningTime="2026-07-15 18:18:15.449367 +0000 UTC m=+76.059180187" Jul 15 18:18:16.433522 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.433480 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnm4m" event={"ID":"38192496-f9a3-42ff-88db-26a2d9452e2a","Type":"ContainerStarted","Data":"1dbec809c91b73d74e41b0ab63f9d4c772b653a144bb7f8111e3d4139b502dee"} Jul 15 18:18:16.435007 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.434983 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" event={"ID":"8dfdf9af-ead0-4cdd-bc39-6a65e5b40aeb","Type":"ContainerStarted","Data":"6031d2292a80fbaeb3965633c856114beb17475e8fdbff0c56f561e01594b2ab"} Jul 15 18:18:16.436345 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.436322 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h9rmb" event={"ID":"6aaeba4e-5969-4d6a-992d-4ddfaf2dedb1","Type":"ContainerStarted","Data":"d819ef808de4189cc466a2037ce8bb653ca173590ac557baf66cc04d44413fee"} Jul 15 18:18:16.436457 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.436367 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:18:16.437905 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.437884 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2n6fz" event={"ID":"19672c70-677e-46a2-9a56-399fc656d1a5","Type":"ContainerStarted","Data":"abb7479f3946729ebcbd21c1e4b34d5e3a2bdc7fcdef3f1a3879b00ed3b82aaa"} Jul 15 18:18:16.439826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.439801 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx5wd" event={"ID":"34cf3357-ec56-4e44-8b1a-58379395479b","Type":"ContainerStarted","Data":"2fa3f4ba74f73a7457d785618520e5df48deeed2b13d39da1cf47f8d3726fbe9"} Jul 15 18:18:16.439910 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.439865 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xx5wd" Jul 15 18:18:16.439910 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.439890 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx5wd" event={"ID":"34cf3357-ec56-4e44-8b1a-58379395479b","Type":"ContainerStarted","Data":"d5333906c2a2d773b6065f51c64c38410b327f1631423fbb42ff54350af2c109"} Jul 15 18:18:16.441855 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.441825 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" event={"ID":"320bfd49-dfcb-45c4-9589-aa82688b0140","Type":"ContainerStarted","Data":"03a61cbcab296d43ead65216d013452c65f3be43db53f3e078945a0e168595a5"} Jul 15 18:18:16.441947 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.441861 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" event={"ID":"320bfd49-dfcb-45c4-9589-aa82688b0140","Type":"ContainerStarted","Data":"06f2986ee5d310c26ad060097d93bceea4a1a67675797346ebabf53fe91c29d3"} Jul 15 18:18:16.441947 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.441880 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" event={"ID":"320bfd49-dfcb-45c4-9589-aa82688b0140","Type":"ContainerStarted","Data":"b5f3fe3a9c3d786802ad64d6a5fa751fb77cfa7643040b939fe139f026dd3b51"} Jul 15 18:18:16.443301 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.443277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" event={"ID":"305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9","Type":"ContainerStarted","Data":"6236b6c2d20613b85311942765e80b070aea14263a57a741b9fde6b7c681659e"} Jul 15 18:18:16.445142 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.445117 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-csskv" event={"ID":"97944bdc-1dea-4012-ac18-4827adcd2da6","Type":"ContainerStarted","Data":"a336ba85cd3166c505fe222b513fee9c4d55c86c50d7bb9e4458656d22ac3d3a"} Jul 15 18:18:16.445234 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.445160 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-csskv" event={"ID":"97944bdc-1dea-4012-ac18-4827adcd2da6","Type":"ContainerStarted","Data":"11052673277a3136f4386b75c655db7842a29799a7a25260f4a2fda34d21817e"} Jul 15 18:18:16.451500 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.451466 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qnm4m" podStartSLOduration=18.456772129 podStartE2EDuration="38.451455289s" totalImagesPullingTime="19.99468316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.450427801 +0000 UTC m=+77.060240990" watchObservedRunningTime="2026-07-15 18:18:16.451455289 +0000 UTC m=+77.061268476" Jul 15 18:18:16.469328 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.469276 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xx5wd" podStartSLOduration=40.376943176 podStartE2EDuration="43.469262975s" totalImagesPullingTime="3.092319799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.468655278 +0000 UTC m=+77.078468466" watchObservedRunningTime="2026-07-15 18:18:16.469262975 +0000 UTC m=+77.079076164" Jul 15 18:18:16.487053 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.487004 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66959d9bd8-q2lwx" podStartSLOduration=3.036739165 podStartE2EDuration="13.486987879s" totalImagesPullingTime="10.450248714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.486554661 +0000 UTC m=+77.096367852" watchObservedRunningTime="2026-07-15 18:18:16.486987879 +0000 UTC m=+77.096801067" Jul 15 18:18:16.510122 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.510069 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-tzkcr" podStartSLOduration=4.583862781 podStartE2EDuration="13.510052764s" totalImagesPullingTime="8.926189983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.509010648 +0000 UTC m=+77.118823839" watchObservedRunningTime="2026-07-15 18:18:16.510052764 +0000 UTC m=+77.119865951" Jul 15 18:18:16.529518 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.529350 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-676f6ddb-x2qqt" podStartSLOduration=2.378720742 podStartE2EDuration="13.529327237s" totalImagesPullingTime="11.150606495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.529007895 +0000 UTC m=+77.138821084" watchObservedRunningTime="2026-07-15 18:18:16.529327237 +0000 UTC m=+77.139140425" Jul 15 18:18:16.545708 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.545659 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h9rmb" podStartSLOduration=66.548408928 podStartE2EDuration="1m16.545645007s" totalImagesPullingTime="9.997236079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.544805909 +0000 UTC m=+77.154619098" watchObservedRunningTime="2026-07-15 18:18:16.545645007 +0000 UTC m=+77.155458194" Jul 15 18:18:16.570215 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.570161 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-csskv" podStartSLOduration=67.064935175 podStartE2EDuration="1m16.570146409s" totalImagesPullingTime="9.505211234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.569055352 +0000 UTC m=+77.178868536" watchObservedRunningTime="2026-07-15 18:18:16.570146409 +0000 UTC m=+77.179959596" Jul 15 18:18:16.586110 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:16.586062 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2n6fz" podStartSLOduration=66.508127111 podStartE2EDuration="1m16.586046205s" totalImagesPullingTime="10.077919094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 18:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:16.585481565 +0000 UTC m=+77.195294744" watchObservedRunningTime="2026-07-15 18:18:16.586046205 +0000 UTC m=+77.195859449" Jul 15 18:18:25.722302 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:25.722190 2557 patch_prober.go:28] interesting pod/image-registry-64c86bbcb8-d6ggr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 18:18:25.722302 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:25.722255 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" podUID="c8bba59a-4efb-41f8-817b-8a04c1bee4b6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 18:18:26.455052 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:26.455017 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xx5wd" Jul 15 18:18:28.913323 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.913282 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f8cpq"] Jul 15 18:18:28.916996 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.916970 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.919841 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.919812 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 18:18:28.919976 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.919898 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 18:18:28.920041 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.920015 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wmw6k\"" Jul 15 18:18:28.920298 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.920115 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 18:18:28.920298 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.920139 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 18:18:28.920876 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.920854 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 18:18:28.921027 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.921011 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 18:18:28.987809 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.987748 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-root\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988022 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.987935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988022 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.987981 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-accelerators-collector-config\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988022 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.988012 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-textfile\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988166 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.988084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4m9t\" (UniqueName: \"kubernetes.io/projected/28d1c725-9a43-42f5-9439-43ddb744be24-kube-api-access-x4m9t\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988202 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.988155 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-wtmp\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988238 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.988206 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-tls\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988271 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.988232 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28d1c725-9a43-42f5-9439-43ddb744be24-metrics-client-ca\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:28.988271 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:28.988256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-sys\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089549 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089512 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-tls\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089549 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28d1c725-9a43-42f5-9439-43ddb744be24-metrics-client-ca\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-sys\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-root\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-root\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:18:29.089674 2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089706 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-sys\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089723 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:18:29.089751 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-tls podName:28d1c725-9a43-42f5-9439-43ddb744be24 nodeName:}" failed. No retries permitted until 2026-07-15 18:18:29.58973356 +0000 UTC m=+90.199546729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-tls") pod "node-exporter-f8cpq" (UID: "28d1c725-9a43-42f5-9439-43ddb744be24") : secret "node-exporter-tls" not found Jul 15 18:18:29.089807 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089809 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-accelerators-collector-config\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-textfile\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089855 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4m9t\" (UniqueName: \"kubernetes.io/projected/28d1c725-9a43-42f5-9439-43ddb744be24-kube-api-access-x4m9t\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.089963 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-wtmp\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.090135 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-wtmp\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.090154 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-textfile\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.090220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28d1c725-9a43-42f5-9439-43ddb744be24-metrics-client-ca\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.090472 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.090304 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-accelerators-collector-config\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.091991 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.091968 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.100041 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.100017 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4m9t\" (UniqueName: \"kubernetes.io/projected/28d1c725-9a43-42f5-9439-43ddb744be24-kube-api-access-x4m9t\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.593948 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.593903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-tls\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.596196 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.596177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/28d1c725-9a43-42f5-9439-43ddb744be24-node-exporter-tls\") pod \"node-exporter-f8cpq\" (UID: \"28d1c725-9a43-42f5-9439-43ddb744be24\") " pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.826071 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:29.826039 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f8cpq" Jul 15 18:18:29.839393 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:18:29.839363 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d1c725_9a43_42f5_9439_43ddb744be24.slice/crio-17ed6a461b92830c799fab7226e18e95139bcd9e971db7dc001d40c0b35021a9 WatchSource:0}: Error finding container 17ed6a461b92830c799fab7226e18e95139bcd9e971db7dc001d40c0b35021a9: Status 404 returned error can't find the container with id 17ed6a461b92830c799fab7226e18e95139bcd9e971db7dc001d40c0b35021a9 Jul 15 18:18:30.501263 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:30.501215 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8cpq" event={"ID":"28d1c725-9a43-42f5-9439-43ddb744be24","Type":"ContainerStarted","Data":"17ed6a461b92830c799fab7226e18e95139bcd9e971db7dc001d40c0b35021a9"} Jul 15 18:18:31.506933 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:31.506845 2557 generic.go:356] "Generic (PLEG): container finished" podID="28d1c725-9a43-42f5-9439-43ddb744be24" containerID="e31e1e79293cf63728d84f758ba80f415738af8008e94650022c3f6d6f1471c5" exitCode=0 Jul 15 18:18:31.507342 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:31.506944 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8cpq" event={"ID":"28d1c725-9a43-42f5-9439-43ddb744be24","Type":"ContainerDied","Data":"e31e1e79293cf63728d84f758ba80f415738af8008e94650022c3f6d6f1471c5"} Jul 15 18:18:32.512285 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:32.512247 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8cpq" event={"ID":"28d1c725-9a43-42f5-9439-43ddb744be24","Type":"ContainerStarted","Data":"4a41cee64565e2e31be2f87549a65631fe0c146bf9afce9c5ece6051fb99fca1"} Jul 15 18:18:32.512655 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:32.512314 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8cpq" event={"ID":"28d1c725-9a43-42f5-9439-43ddb744be24","Type":"ContainerStarted","Data":"042a33c75a7998ab960236e1be0bbaff5a241141620daae94256afedd1648637"} Jul 15 18:18:32.530378 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:32.530321 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f8cpq" podStartSLOduration=3.110026759 podStartE2EDuration="4.530303113s" totalImagesPullingTime="1.342089496s" totalInitContainerRuntime="78.186858ms" isStatefulPod=true podCreationTimestamp="2026-07-15 18:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:18:32.529391116 +0000 UTC m=+93.139204308" watchObservedRunningTime="2026-07-15 18:18:32.530303113 +0000 UTC m=+93.140116328" Jul 15 18:18:37.456382 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:37.456351 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64c86bbcb8-d6ggr" Jul 15 18:18:45.559908 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:45.559877 2557 generic.go:356] "Generic (PLEG): container finished" podID="7f08fedb-4c6a-43eb-86d5-4a56139355a2" containerID="54bdb9583530d878d30dfd1b7d0277f79b1bca9c03ba018eee418def519b6c8d" exitCode=0 Jul 15 18:18:45.560378 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:45.559936 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" event={"ID":"7f08fedb-4c6a-43eb-86d5-4a56139355a2","Type":"ContainerDied","Data":"54bdb9583530d878d30dfd1b7d0277f79b1bca9c03ba018eee418def519b6c8d"} Jul 15 18:18:45.560378 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:45.560319 2557 scope.go:117] "RemoveContainer" containerID="54bdb9583530d878d30dfd1b7d0277f79b1bca9c03ba018eee418def519b6c8d" Jul 15 18:18:46.564737 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:46.564704 2557 generic.go:356] "Generic (PLEG): container finished" podID="ea3d3efe-2f9f-4b75-9521-e95e7d1b624a" containerID="e889dbdf9b7d67e9993312d7aaefd97d6b5a0a9ffa900bdf953cfa7f642bd7c8" exitCode=0 Jul 15 18:18:46.565183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:46.564792 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" event={"ID":"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a","Type":"ContainerDied","Data":"e889dbdf9b7d67e9993312d7aaefd97d6b5a0a9ffa900bdf953cfa7f642bd7c8"} Jul 15 18:18:46.565183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:46.565106 2557 scope.go:117] "RemoveContainer" containerID="e889dbdf9b7d67e9993312d7aaefd97d6b5a0a9ffa900bdf953cfa7f642bd7c8" Jul 15 18:18:46.566735 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:46.566712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wrgpz" event={"ID":"7f08fedb-4c6a-43eb-86d5-4a56139355a2","Type":"ContainerStarted","Data":"bf031812cbcbc7291f35009a219fe7d7ead99d390f1246402db204d3b262f39d"} Jul 15 18:18:47.454194 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:47.454162 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h9rmb" Jul 15 18:18:47.572129 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:18:47.572097 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-4pthj" event={"ID":"ea3d3efe-2f9f-4b75-9521-e95e7d1b624a","Type":"ContainerStarted","Data":"bddf661c46237f56b0f0d753ebfd70da35a022dcb2d5359fb17a37c5308d2728"} Jul 15 18:21:59.026526 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.026488 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5"] Jul 15 18:21:59.028627 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.028605 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.031547 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.031522 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 18:21:59.031667 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.031569 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 18:21:59.031667 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.031575 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 18:21:59.031882 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.031869 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 18:21:59.031940 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.031922 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-555sq\"" Jul 15 18:21:59.057357 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.057317 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5"] Jul 15 18:21:59.172632 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.172594 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.172861 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.172711 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-manifests\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.172861 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.172740 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.172861 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.172791 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-tmp\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.173011 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.172890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45s4l\" (UniqueName: \"kubernetes.io/projected/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-kube-api-access-45s4l\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.273420 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.273381 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-manifests\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.273420 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.273421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.273675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.273442 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-tmp\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.273675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.273486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45s4l\" (UniqueName: \"kubernetes.io/projected/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-kube-api-access-45s4l\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.273675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.273514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.273954 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.273928 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-manifests\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.275832 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.275808 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-tmp\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.276055 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.276038 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.276178 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.276159 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.281603 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.281544 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45s4l\" (UniqueName: \"kubernetes.io/projected/e4b7adc6-3fd2-4930-b357-2ca42676d8ce-kube-api-access-45s4l\") pod \"opendatahub-operator-controller-manager-5664889d69-wsdn5\" (UID: \"e4b7adc6-3fd2-4930-b357-2ca42676d8ce\") " pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.339580 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.339538 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:21:59.476652 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.476620 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5"] Jul 15 18:21:59.480730 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:21:59.480701 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b7adc6_3fd2_4930_b357_2ca42676d8ce.slice/crio-126eca5d3e31d242c3dcc446be79f786497dbd3fa6e24c9dd4c165c712149801 WatchSource:0}: Error finding container 126eca5d3e31d242c3dcc446be79f786497dbd3fa6e24c9dd4c165c712149801: Status 404 returned error can't find the container with id 126eca5d3e31d242c3dcc446be79f786497dbd3fa6e24c9dd4c165c712149801 Jul 15 18:21:59.804585 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.804555 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:21:59.809633 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.809608 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:21:59.812212 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.812187 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:21:59.816727 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.816705 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:21:59.816929 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:21:59.816895 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 18:22:00.249264 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:00.249219 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" event={"ID":"e4b7adc6-3fd2-4930-b357-2ca42676d8ce","Type":"ContainerStarted","Data":"126eca5d3e31d242c3dcc446be79f786497dbd3fa6e24c9dd4c165c712149801"} Jul 15 18:22:03.266092 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:03.266054 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" event={"ID":"e4b7adc6-3fd2-4930-b357-2ca42676d8ce","Type":"ContainerStarted","Data":"2ae7812dacaf0100371e15e277f365bc2475a9b7a0f59ee7e36276a040a71421"} Jul 15 18:22:04.270976 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:04.270943 2557 generic.go:356] "Generic (PLEG): container finished" podID="e4b7adc6-3fd2-4930-b357-2ca42676d8ce" containerID="2ae7812dacaf0100371e15e277f365bc2475a9b7a0f59ee7e36276a040a71421" exitCode=0 Jul 15 18:22:04.271337 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:04.271022 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" event={"ID":"e4b7adc6-3fd2-4930-b357-2ca42676d8ce","Type":"ContainerDied","Data":"2ae7812dacaf0100371e15e277f365bc2475a9b7a0f59ee7e36276a040a71421"} Jul 15 18:22:04.271488 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:04.271473 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 18:22:05.277124 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:05.277085 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" event={"ID":"e4b7adc6-3fd2-4930-b357-2ca42676d8ce","Type":"ContainerStarted","Data":"5e688fb02e474ad0d0b7b047f7eb9c52a5c009b02abff8af9f16678b65e2b5f6"} Jul 15 18:22:05.277511 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:05.277192 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:22:05.299286 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:05.299231 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" podStartSLOduration=1.9707140010000002 podStartE2EDuration="6.299213571s" totalImagesPullingTime="3.320529348s" totalInitContainerRuntime="1.007970222s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:21:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:22:05.29839092 +0000 UTC m=+305.908204107" watchObservedRunningTime="2026-07-15 18:22:05.299213571 +0000 UTC m=+305.909026758" Jul 15 18:22:16.283731 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:16.283698 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5664889d69-wsdn5" Jul 15 18:22:19.407572 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.407537 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g6bh4"] Jul 15 18:22:19.416827 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.416807 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:19.421014 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.420979 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 18:22:19.421136 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.421052 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6j4h7\"" Jul 15 18:22:19.425175 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.424641 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-564c7\" (UniqueName: \"kubernetes.io/projected/c97fb344-f922-4fa9-adb5-4f6a5498673d-kube-api-access-564c7\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:19.425175 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.424707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97fb344-f922-4fa9-adb5-4f6a5498673d-cert\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:19.431945 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.431923 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g6bh4"] Jul 15 18:22:19.525413 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.525378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-564c7\" (UniqueName: \"kubernetes.io/projected/c97fb344-f922-4fa9-adb5-4f6a5498673d-kube-api-access-564c7\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:19.525413 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.525416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97fb344-f922-4fa9-adb5-4f6a5498673d-cert\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:19.525617 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:22:19.525512 2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 18:22:19.525617 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:22:19.525576 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97fb344-f922-4fa9-adb5-4f6a5498673d-cert podName:c97fb344-f922-4fa9-adb5-4f6a5498673d nodeName:}" failed. No retries permitted until 2026-07-15 18:22:20.025560802 +0000 UTC m=+320.635373968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c97fb344-f922-4fa9-adb5-4f6a5498673d-cert") pod "odh-model-controller-7998c67dc4-g6bh4" (UID: "c97fb344-f922-4fa9-adb5-4f6a5498673d") : secret "odh-model-controller-webhook-cert" not found Jul 15 18:22:19.569141 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.569100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-564c7\" (UniqueName: \"kubernetes.io/projected/c97fb344-f922-4fa9-adb5-4f6a5498673d-kube-api-access-564c7\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:19.648228 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.648194 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz"] Jul 15 18:22:19.650966 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.650948 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.662043 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.662019 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wpdlm\"" Jul 15 18:22:19.662173 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.662019 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 18:22:19.662264 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.662247 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 18:22:19.662990 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.662976 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 18:22:19.663036 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.662982 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 18:22:19.663080 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.663064 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 18:22:19.680178 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.680146 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz"] Jul 15 18:22:19.727152 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.727113 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c43dbd64-9f18-433e-a88b-d3b491c0eea2-cert\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.727344 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.727174 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c43dbd64-9f18-433e-a88b-d3b491c0eea2-manager-config\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.727344 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.727201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb8qt\" (UniqueName: \"kubernetes.io/projected/c43dbd64-9f18-433e-a88b-d3b491c0eea2-kube-api-access-cb8qt\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.727344 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.727235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c43dbd64-9f18-433e-a88b-d3b491c0eea2-metrics-cert\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.827908 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.827866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c43dbd64-9f18-433e-a88b-d3b491c0eea2-cert\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.827908 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.827917 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c43dbd64-9f18-433e-a88b-d3b491c0eea2-manager-config\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.828163 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.827941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb8qt\" (UniqueName: \"kubernetes.io/projected/c43dbd64-9f18-433e-a88b-d3b491c0eea2-kube-api-access-cb8qt\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.828163 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.827979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c43dbd64-9f18-433e-a88b-d3b491c0eea2-metrics-cert\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.828688 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.828664 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c43dbd64-9f18-433e-a88b-d3b491c0eea2-manager-config\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.830310 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.830283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c43dbd64-9f18-433e-a88b-d3b491c0eea2-cert\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.830404 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.830348 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c43dbd64-9f18-433e-a88b-d3b491c0eea2-metrics-cert\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.838098 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.838076 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb8qt\" (UniqueName: \"kubernetes.io/projected/c43dbd64-9f18-433e-a88b-d3b491c0eea2-kube-api-access-cb8qt\") pod \"lws-controller-manager-68b4879c65-r8zwz\" (UID: \"c43dbd64-9f18-433e-a88b-d3b491c0eea2\") " pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:19.960443 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:19.960360 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:20.030154 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:20.030116 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97fb344-f922-4fa9-adb5-4f6a5498673d-cert\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:20.032749 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:20.032701 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c97fb344-f922-4fa9-adb5-4f6a5498673d-cert\") pod \"odh-model-controller-7998c67dc4-g6bh4\" (UID: \"c97fb344-f922-4fa9-adb5-4f6a5498673d\") " pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:20.111042 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:20.111015 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz"] Jul 15 18:22:20.111833 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:22:20.111811 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43dbd64_9f18_433e_a88b_d3b491c0eea2.slice/crio-1015ec5ec113e9c734b8bc200a87d11be7ec8f22026d598e543e14b5d753732d WatchSource:0}: Error finding container 1015ec5ec113e9c734b8bc200a87d11be7ec8f22026d598e543e14b5d753732d: Status 404 returned error can't find the container with id 1015ec5ec113e9c734b8bc200a87d11be7ec8f22026d598e543e14b5d753732d Jul 15 18:22:20.329083 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:20.328993 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:20.338278 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:20.338249 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" event={"ID":"c43dbd64-9f18-433e-a88b-d3b491c0eea2","Type":"ContainerStarted","Data":"1015ec5ec113e9c734b8bc200a87d11be7ec8f22026d598e543e14b5d753732d"} Jul 15 18:22:20.473231 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:20.473205 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g6bh4"] Jul 15 18:22:20.475406 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:22:20.475367 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97fb344_f922_4fa9_adb5_4f6a5498673d.slice/crio-e5d64a442dd63f098d3a67aad72f0d9b8fd3df28f726d545df9bb2630508c28d WatchSource:0}: Error finding container e5d64a442dd63f098d3a67aad72f0d9b8fd3df28f726d545df9bb2630508c28d: Status 404 returned error can't find the container with id e5d64a442dd63f098d3a67aad72f0d9b8fd3df28f726d545df9bb2630508c28d Jul 15 18:22:21.345416 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:21.345376 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" event={"ID":"c97fb344-f922-4fa9-adb5-4f6a5498673d","Type":"ContainerStarted","Data":"e5d64a442dd63f098d3a67aad72f0d9b8fd3df28f726d545df9bb2630508c28d"} Jul 15 18:22:24.362820 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:24.362746 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" event={"ID":"c43dbd64-9f18-433e-a88b-d3b491c0eea2","Type":"ContainerStarted","Data":"30323a1639c4ac7b936d142c419de89473797e4537bb417f38a8a0c0e7b5f4d0"} Jul 15 18:22:24.363287 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:24.362943 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:24.364304 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:24.364268 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" event={"ID":"c97fb344-f922-4fa9-adb5-4f6a5498673d","Type":"ContainerStarted","Data":"b1fd0bf67f726b046cfae3433893aa6506323623aede43c3f364909487e5441d"} Jul 15 18:22:24.364400 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:24.364321 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:22:24.381160 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:24.381114 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" podStartSLOduration=1.667429726 podStartE2EDuration="5.381097344s" totalImagesPullingTime="3.713667618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:22:24.378930832 +0000 UTC m=+324.988744020" watchObservedRunningTime="2026-07-15 18:22:24.381097344 +0000 UTC m=+324.990910533" Jul 15 18:22:24.395398 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:24.395337 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" podStartSLOduration=2.00370954 podStartE2EDuration="5.395316561s" totalImagesPullingTime="3.391607021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:22:24.395282575 +0000 UTC m=+325.005095763" watchObservedRunningTime="2026-07-15 18:22:24.395316561 +0000 UTC m=+325.005129750" Jul 15 18:22:26.503229 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.503189 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj"] Jul 15 18:22:26.506528 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.506505 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.509095 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.509075 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ttrjp\"" Jul 15 18:22:26.510935 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.510918 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 18:22:26.511095 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.511079 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 18:22:26.512081 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.512061 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 18:22:26.523290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.523267 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj"] Jul 15 18:22:26.589874 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.589844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4db6401-7924-468b-98a8-16288d2304ee-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.589884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.589905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px8x4\" (UniqueName: \"kubernetes.io/projected/b4db6401-7924-468b-98a8-16288d2304ee-kube-api-access-px8x4\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.589961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.590004 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.590066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4db6401-7924-468b-98a8-16288d2304ee-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.590105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.590136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.590404 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.590177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4db6401-7924-468b-98a8-16288d2304ee-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691093 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4db6401-7924-468b-98a8-16288d2304ee-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691101 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691130 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px8x4\" (UniqueName: \"kubernetes.io/projected/b4db6401-7924-468b-98a8-16288d2304ee-kube-api-access-px8x4\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691182 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4db6401-7924-468b-98a8-16288d2304ee-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691290 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691587 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691290 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691587 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4db6401-7924-468b-98a8-16288d2304ee-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691587 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691534 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691587 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691564 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691743 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691643 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691808 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691741 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.691874 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.691852 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4db6401-7924-468b-98a8-16288d2304ee-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.693629 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.693611 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4db6401-7924-468b-98a8-16288d2304ee-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.693991 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.693969 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4db6401-7924-468b-98a8-16288d2304ee-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.699610 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.699588 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4db6401-7924-468b-98a8-16288d2304ee-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.699681 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.699599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px8x4\" (UniqueName: \"kubernetes.io/projected/b4db6401-7924-468b-98a8-16288d2304ee-kube-api-access-px8x4\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786slhj\" (UID: \"b4db6401-7924-468b-98a8-16288d2304ee\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.817094 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.817010 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:26.958581 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:26.958547 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj"] Jul 15 18:22:26.961559 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:22:26.961517 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4db6401_7924_468b_98a8_16288d2304ee.slice/crio-da0e60c7d8ffe2b123d68a7b3ab07eecedc91bfb435ceda1eafdb621f9875d74 WatchSource:0}: Error finding container da0e60c7d8ffe2b123d68a7b3ab07eecedc91bfb435ceda1eafdb621f9875d74: Status 404 returned error can't find the container with id da0e60c7d8ffe2b123d68a7b3ab07eecedc91bfb435ceda1eafdb621f9875d74 Jul 15 18:22:27.377443 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:27.377406 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" event={"ID":"b4db6401-7924-468b-98a8-16288d2304ee","Type":"ContainerStarted","Data":"da0e60c7d8ffe2b123d68a7b3ab07eecedc91bfb435ceda1eafdb621f9875d74"} Jul 15 18:22:29.218421 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.218372 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw"] Jul 15 18:22:29.224779 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.224741 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.228930 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.228903 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 18:22:29.229340 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.229317 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-52q9d\"" Jul 15 18:22:29.229471 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.229429 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 18:22:29.229537 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.229472 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 18:22:29.257038 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.257004 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw"] Jul 15 18:22:29.311479 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311441 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.311654 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311487 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.311654 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311514 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsdbx\" (UniqueName: \"kubernetes.io/projected/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-kube-api-access-nsdbx\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.311654 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.311836 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311655 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.311836 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311682 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.311836 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.311727 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.412601 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.412818 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.412818 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.412818 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412719 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsdbx\" (UniqueName: \"kubernetes.io/projected/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-kube-api-access-nsdbx\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.412818 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412752 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.412818 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.413083 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.412839 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.413609 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.413563 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.415400 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.415330 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.415511 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.415455 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.415680 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.415638 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.415926 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.415908 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.427416 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.427382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsdbx\" (UniqueName: \"kubernetes.io/projected/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-kube-api-access-nsdbx\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.430058 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.430034 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/987f8e4b-7f84-4201-bff9-1475aa8dc4c3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-mjqlw\" (UID: \"987f8e4b-7f84-4201-bff9-1475aa8dc4c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.535668 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.535581 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:29.938396 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.938348 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:22:29.938500 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.938449 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:22:29.938500 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:29.938494 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:22:30.026709 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:30.026676 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw"] Jul 15 18:22:30.030149 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:22:30.030113 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987f8e4b_7f84_4201_bff9_1475aa8dc4c3.slice/crio-fb9686fffd0490c56c87322d45cecaec85d1769f47c109e2a6ddda1701597bc9 WatchSource:0}: Error finding container fb9686fffd0490c56c87322d45cecaec85d1769f47c109e2a6ddda1701597bc9: Status 404 returned error can't find the container with id fb9686fffd0490c56c87322d45cecaec85d1769f47c109e2a6ddda1701597bc9 Jul 15 18:22:30.391730 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:30.391694 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" event={"ID":"987f8e4b-7f84-4201-bff9-1475aa8dc4c3","Type":"ContainerStarted","Data":"fb9686fffd0490c56c87322d45cecaec85d1769f47c109e2a6ddda1701597bc9"} Jul 15 18:22:30.393236 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:30.393207 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" event={"ID":"b4db6401-7924-468b-98a8-16288d2304ee","Type":"ContainerStarted","Data":"4442eca5facbdd5eba9ecfe69303ae7a6a5b8016b4a6fbf279ecf8622a8b1358"} Jul 15 18:22:30.456597 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:30.456548 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" podStartSLOduration=1.48258527 podStartE2EDuration="4.456529137s" totalImagesPullingTime="2.973943867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:22:30.45498932 +0000 UTC m=+331.064802508" watchObservedRunningTime="2026-07-15 18:22:30.456529137 +0000 UTC m=+331.066342325" Jul 15 18:22:30.818058 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:30.817970 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:31.823051 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:31.823014 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:32.404549 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:32.404517 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:32.405583 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:32.405554 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786slhj" Jul 15 18:22:32.626382 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:32.626349 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:22:32.626454 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:32.626414 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:22:33.410301 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:33.410263 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" event={"ID":"987f8e4b-7f84-4201-bff9-1475aa8dc4c3","Type":"ContainerStarted","Data":"4b39f0f64d2df7318237a5433432d33d7b184678a729dd51cf4c925c837ed7f4"} Jul 15 18:22:33.410704 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:33.410512 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:33.412155 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:33.412129 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-mjqlw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 18:22:33.412283 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:33.412182 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" podUID="987f8e4b-7f84-4201-bff9-1475aa8dc4c3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 18:22:33.432723 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:33.432673 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" podStartSLOduration=1.839030602 podStartE2EDuration="4.4326482s" totalImagesPullingTime="2.593617598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:22:33.430980667 +0000 UTC m=+334.040793870" watchObservedRunningTime="2026-07-15 18:22:33.4326482 +0000 UTC m=+334.042461388" Jul 15 18:22:34.415604 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:34.415580 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mjqlw" Jul 15 18:22:35.370258 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:35.370229 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68b4879c65-r8zwz" Jul 15 18:22:35.370458 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:22:35.370306 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-g6bh4" Jul 15 18:26:16.102500 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.102463 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j"] Jul 15 18:26:16.105621 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.105597 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.108203 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.108184 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 18:26:16.108314 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.108230 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nk64h\"" Jul 15 18:26:16.116449 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.116427 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j"] Jul 15 18:26:16.154011 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.153974 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjvbv\" (UniqueName: \"kubernetes.io/projected/7eb63288-7bb5-45c4-8e1f-4a416d1d5306-kube-api-access-jjvbv\") pod \"payload-pre-processing-74584bb9f8-mtl2j\" (UID: \"7eb63288-7bb5-45c4-8e1f-4a416d1d5306\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.154183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.154056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb63288-7bb5-45c4-8e1f-4a416d1d5306-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-mtl2j\" (UID: \"7eb63288-7bb5-45c4-8e1f-4a416d1d5306\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.255047 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.255012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjvbv\" (UniqueName: \"kubernetes.io/projected/7eb63288-7bb5-45c4-8e1f-4a416d1d5306-kube-api-access-jjvbv\") pod \"payload-pre-processing-74584bb9f8-mtl2j\" (UID: \"7eb63288-7bb5-45c4-8e1f-4a416d1d5306\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.255213 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.255067 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb63288-7bb5-45c4-8e1f-4a416d1d5306-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-mtl2j\" (UID: \"7eb63288-7bb5-45c4-8e1f-4a416d1d5306\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.255708 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.255689 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eb63288-7bb5-45c4-8e1f-4a416d1d5306-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-mtl2j\" (UID: \"7eb63288-7bb5-45c4-8e1f-4a416d1d5306\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.263501 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.263474 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjvbv\" (UniqueName: \"kubernetes.io/projected/7eb63288-7bb5-45c4-8e1f-4a416d1d5306-kube-api-access-jjvbv\") pod \"payload-pre-processing-74584bb9f8-mtl2j\" (UID: \"7eb63288-7bb5-45c4-8e1f-4a416d1d5306\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.415993 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.415913 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:16.548779 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:16.548743 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j"] Jul 15 18:26:16.550681 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:26:16.550658 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb63288_7bb5_45c4_8e1f_4a416d1d5306.slice/crio-d09bdc1ee81c6cf1c8c4c4a452cbb37839963601b9b2403fd815de8e13a99fb1 WatchSource:0}: Error finding container d09bdc1ee81c6cf1c8c4c4a452cbb37839963601b9b2403fd815de8e13a99fb1: Status 404 returned error can't find the container with id d09bdc1ee81c6cf1c8c4c4a452cbb37839963601b9b2403fd815de8e13a99fb1 Jul 15 18:26:17.313296 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:17.313259 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" event={"ID":"7eb63288-7bb5-45c4-8e1f-4a416d1d5306","Type":"ContainerStarted","Data":"d09bdc1ee81c6cf1c8c4c4a452cbb37839963601b9b2403fd815de8e13a99fb1"} Jul 15 18:26:18.318791 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:18.318732 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" event={"ID":"7eb63288-7bb5-45c4-8e1f-4a416d1d5306","Type":"ContainerStarted","Data":"1fe08402793e21a7d415b615ff758c7cefd5ec8fcd7c8f41542c89ae24d0f05c"} Jul 15 18:26:18.319164 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:18.318896 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:18.337082 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:18.337016 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" podStartSLOduration=1.151845699 podStartE2EDuration="2.336999571s" totalImagesPullingTime="1.185153872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:26:18.335828024 +0000 UTC m=+558.945641209" watchObservedRunningTime="2026-07-15 18:26:18.336999571 +0000 UTC m=+558.946812758" Jul 15 18:26:29.325704 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:29.325672 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-mtl2j" Jul 15 18:26:59.838289 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:59.838263 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:26:59.838906 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:59.838888 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:26:59.844078 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:59.844058 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:26:59.844530 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:26:59.844515 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:27:29.302465 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.302396 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv"] Jul 15 18:27:29.304602 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.304586 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.308174 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.308152 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8r5n2\"" Jul 15 18:27:29.308742 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.308725 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 18:27:29.309167 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.309148 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 18:27:29.309888 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.309873 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 18:27:29.335454 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.335426 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv"] Jul 15 18:27:29.459596 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.459565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.459596 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.459598 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbk2s\" (UniqueName: \"kubernetes.io/projected/f1c44ea9-96f7-4f70-899c-cd45ef545d69-kube-api-access-nbk2s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.459890 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.459630 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.459890 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.459677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.459890 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.459760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.459890 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.459812 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c44ea9-96f7-4f70-899c-cd45ef545d69-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.560760 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.560650 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.560760 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.560702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.560760 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.560754 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.560760 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.560799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c44ea9-96f7-4f70-899c-cd45ef545d69-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.561116 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.560860 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.561116 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.560885 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbk2s\" (UniqueName: \"kubernetes.io/projected/f1c44ea9-96f7-4f70-899c-cd45ef545d69-kube-api-access-nbk2s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.561243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.561217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.561293 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.561259 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.561355 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.561303 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.563171 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.563140 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1c44ea9-96f7-4f70-899c-cd45ef545d69-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.563330 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.563315 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1c44ea9-96f7-4f70-899c-cd45ef545d69-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.571470 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.571447 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbk2s\" (UniqueName: \"kubernetes.io/projected/f1c44ea9-96f7-4f70-899c-cd45ef545d69-kube-api-access-nbk2s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8x4nv\" (UID: \"f1c44ea9-96f7-4f70-899c-cd45ef545d69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.614597 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.614567 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:29.755539 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.755516 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv"] Jul 15 18:27:29.757749 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:27:29.757722 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c44ea9_96f7_4f70_899c_cd45ef545d69.slice/crio-cf87a2b4bd157a57224f75107b05e0d9ae2f1e340837b282a201ea6b1b225e74 WatchSource:0}: Error finding container cf87a2b4bd157a57224f75107b05e0d9ae2f1e340837b282a201ea6b1b225e74: Status 404 returned error can't find the container with id cf87a2b4bd157a57224f75107b05e0d9ae2f1e340837b282a201ea6b1b225e74 Jul 15 18:27:29.759038 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:29.759023 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 18:27:30.622691 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:30.622647 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" event={"ID":"f1c44ea9-96f7-4f70-899c-cd45ef545d69","Type":"ContainerStarted","Data":"cf87a2b4bd157a57224f75107b05e0d9ae2f1e340837b282a201ea6b1b225e74"} Jul 15 18:27:31.628198 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:31.628145 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" event={"ID":"f1c44ea9-96f7-4f70-899c-cd45ef545d69","Type":"ContainerStarted","Data":"590837c1f6c7438a5971636dcad60c1dfcd762f0064cc672dd50883cf3d959a6"} Jul 15 18:27:39.615763 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:39.615730 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:39.616152 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:39.615813 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:39.628535 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:39.628508 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:27:39.648694 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:39.648640 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" podStartSLOduration=9.676648845999999 podStartE2EDuration="10.648624075s" totalImagesPullingTime="971.975229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:27:31.652662778 +0000 UTC m=+632.262475966" watchObservedRunningTime="2026-07-15 18:27:39.648624075 +0000 UTC m=+640.258437262" Jul 15 18:27:39.674237 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:27:39.674212 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8x4nv" Jul 15 18:28:05.601244 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.601213 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm"] Jul 15 18:28:05.606102 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.606078 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.608797 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.608763 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 18:28:05.615055 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.615031 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm"] Jul 15 18:28:05.666369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.666333 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.666369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.666368 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.666568 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.666383 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.666568 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.666491 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.666568 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.666536 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qch\" (UniqueName: \"kubernetes.io/projected/0c1f34f8-eceb-4216-9e2c-5583332bdea1-kube-api-access-m7qch\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.666677 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.666608 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c1f34f8-eceb-4216-9e2c-5583332bdea1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767377 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qch\" (UniqueName: \"kubernetes.io/projected/0c1f34f8-eceb-4216-9e2c-5583332bdea1-kube-api-access-m7qch\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767406 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c1f34f8-eceb-4216-9e2c-5583332bdea1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767850 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767802 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767901 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.767992 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.767963 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.769733 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.769711 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c1f34f8-eceb-4216-9e2c-5583332bdea1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.770180 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.770159 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c1f34f8-eceb-4216-9e2c-5583332bdea1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.777014 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.776989 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qch\" (UniqueName: \"kubernetes.io/projected/0c1f34f8-eceb-4216-9e2c-5583332bdea1-kube-api-access-m7qch\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm\" (UID: \"0c1f34f8-eceb-4216-9e2c-5583332bdea1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:05.916560 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:05.916461 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:06.048270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:06.048243 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm"] Jul 15 18:28:06.051052 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:28:06.051028 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1f34f8_eceb_4216_9e2c_5583332bdea1.slice/crio-354e9f8153589970a186df869ff471dd53aefe1deae78648e409c072b7a4482c WatchSource:0}: Error finding container 354e9f8153589970a186df869ff471dd53aefe1deae78648e409c072b7a4482c: Status 404 returned error can't find the container with id 354e9f8153589970a186df869ff471dd53aefe1deae78648e409c072b7a4482c Jul 15 18:28:06.779472 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:06.779366 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" event={"ID":"0c1f34f8-eceb-4216-9e2c-5583332bdea1","Type":"ContainerStarted","Data":"f061bac526e60ef2f847a710af1ce77fe7e16e37f011bbccb2d443b18ca0ebdf"} Jul 15 18:28:06.779472 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:06.779423 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" event={"ID":"0c1f34f8-eceb-4216-9e2c-5583332bdea1","Type":"ContainerStarted","Data":"354e9f8153589970a186df869ff471dd53aefe1deae78648e409c072b7a4482c"} Jul 15 18:28:06.800226 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:06.800174 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" podStartSLOduration=1.494427106 podStartE2EDuration="1.800158409s" totalImagesPullingTime="305.731303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:28:06.797949673 +0000 UTC m=+667.407762892" watchObservedRunningTime="2026-07-15 18:28:06.800158409 +0000 UTC m=+667.409971597" Jul 15 18:28:15.917009 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:15.916970 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:15.917453 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:15.917032 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:15.931340 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:15.931310 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:28:16.831887 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:28:16.831849 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm" Jul 15 18:31:59.863457 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:31:59.863363 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:31:59.864549 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:31:59.864529 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:31:59.869951 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:31:59.869929 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:31:59.875794 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:31:59.871149 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:36:59.897884 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:36:59.896236 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:36:59.899252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:36:59.899231 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:36:59.904143 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:36:59.904124 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:36:59.905099 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:36:59.905079 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:41:59.923299 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:41:59.923265 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:41:59.925744 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:41:59.925288 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:41:59.929535 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:41:59.929514 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:41:59.931255 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:41:59.931235 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:43:20.078519 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.078485 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657"] Jul 15 18:43:20.081735 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.081718 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:20.084909 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.084886 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:43:20.084990 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.084886 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:43:20.092158 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.092137 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657"] Jul 15 18:43:20.232637 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.232602 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7983222-91ce-4991-8694-e93bc7cad5b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657\" (UID: \"c7983222-91ce-4991-8694-e93bc7cad5b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:20.333140 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.333045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7983222-91ce-4991-8694-e93bc7cad5b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657\" (UID: \"c7983222-91ce-4991-8694-e93bc7cad5b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:20.333672 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.333652 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7983222-91ce-4991-8694-e93bc7cad5b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657\" (UID: \"c7983222-91ce-4991-8694-e93bc7cad5b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:20.391557 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.391521 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:20.523165 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.523138 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657"] Jul 15 18:43:20.525629 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:43:20.525603 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7983222_91ce_4991_8694_e93bc7cad5b0.slice/crio-e3ef401653b5b0822cfe3e5c1d154b08535d226231a55c9606ea0e5069c3a32a WatchSource:0}: Error finding container e3ef401653b5b0822cfe3e5c1d154b08535d226231a55c9606ea0e5069c3a32a: Status 404 returned error can't find the container with id e3ef401653b5b0822cfe3e5c1d154b08535d226231a55c9606ea0e5069c3a32a Jul 15 18:43:20.527458 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.527442 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 18:43:20.684708 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:20.684675 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" event={"ID":"c7983222-91ce-4991-8694-e93bc7cad5b0","Type":"ContainerStarted","Data":"e3ef401653b5b0822cfe3e5c1d154b08535d226231a55c9606ea0e5069c3a32a"} Jul 15 18:43:23.699934 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:23.699900 2557 generic.go:356] "Generic (PLEG): container finished" podID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerID="7976b23324bd62e4968b4ccdb8986eefbd848efba46dbaf2b45e3616e2c61a04" exitCode=7 Jul 15 18:43:23.700316 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:23.699958 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" event={"ID":"c7983222-91ce-4991-8694-e93bc7cad5b0","Type":"ContainerDied","Data":"7976b23324bd62e4968b4ccdb8986eefbd848efba46dbaf2b45e3616e2c61a04"} Jul 15 18:43:23.700316 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:23.700188 2557 scope.go:117] "RemoveContainer" containerID="7976b23324bd62e4968b4ccdb8986eefbd848efba46dbaf2b45e3616e2c61a04" Jul 15 18:43:24.706487 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:24.706454 2557 generic.go:356] "Generic (PLEG): container finished" podID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerID="43ce444491bc8ca45c12040756c94a63a45eec3b05709a339c7396a431bb1e13" exitCode=7 Jul 15 18:43:24.706900 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:24.706525 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" event={"ID":"c7983222-91ce-4991-8694-e93bc7cad5b0","Type":"ContainerDied","Data":"43ce444491bc8ca45c12040756c94a63a45eec3b05709a339c7396a431bb1e13"} Jul 15 18:43:24.706900 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:24.706588 2557 scope.go:117] "RemoveContainer" containerID="7976b23324bd62e4968b4ccdb8986eefbd848efba46dbaf2b45e3616e2c61a04" Jul 15 18:43:24.706900 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:24.706800 2557 scope.go:117] "RemoveContainer" containerID="43ce444491bc8ca45c12040756c94a63a45eec3b05709a339c7396a431bb1e13" Jul 15 18:43:24.707031 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:43:24.707011 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657_odh-ai-gateway-infra(c7983222-91ce-4991-8694-e93bc7cad5b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" Jul 15 18:43:25.712754 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:25.712729 2557 scope.go:117] "RemoveContainer" containerID="43ce444491bc8ca45c12040756c94a63a45eec3b05709a339c7396a431bb1e13" Jul 15 18:43:25.713127 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:43:25.712928 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657_odh-ai-gateway-infra(c7983222-91ce-4991-8694-e93bc7cad5b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" Jul 15 18:43:40.992453 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:40.992418 2557 scope.go:117] "RemoveContainer" containerID="43ce444491bc8ca45c12040756c94a63a45eec3b05709a339c7396a431bb1e13" Jul 15 18:43:41.792850 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:41.792818 2557 generic.go:356] "Generic (PLEG): container finished" podID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerID="9cc9d20040bbd6dc357a65e3fb7bf367d41c7f569aee258c5d82ca56b0a0a211" exitCode=0 Jul 15 18:43:41.793025 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:41.792896 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" event={"ID":"c7983222-91ce-4991-8694-e93bc7cad5b0","Type":"ContainerDied","Data":"9cc9d20040bbd6dc357a65e3fb7bf367d41c7f569aee258c5d82ca56b0a0a211"} Jul 15 18:43:41.793025 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:41.792961 2557 scope.go:117] "RemoveContainer" containerID="43ce444491bc8ca45c12040756c94a63a45eec3b05709a339c7396a431bb1e13" Jul 15 18:43:42.827457 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:42.827417 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657"] Jul 15 18:43:42.937304 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:42.937279 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:43.023252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.023215 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7983222-91ce-4991-8694-e93bc7cad5b0-maas-api-service-ca\") pod \"c7983222-91ce-4991-8694-e93bc7cad5b0\" (UID: \"c7983222-91ce-4991-8694-e93bc7cad5b0\") " Jul 15 18:43:43.023575 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.023553 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7983222-91ce-4991-8694-e93bc7cad5b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7983222-91ce-4991-8694-e93bc7cad5b0" (UID: "c7983222-91ce-4991-8694-e93bc7cad5b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:43:43.124405 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.124372 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7983222-91ce-4991-8694-e93bc7cad5b0-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:43:43.804618 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.804592 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" Jul 15 18:43:43.804618 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.804610 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657" event={"ID":"c7983222-91ce-4991-8694-e93bc7cad5b0","Type":"ContainerDied","Data":"e3ef401653b5b0822cfe3e5c1d154b08535d226231a55c9606ea0e5069c3a32a"} Jul 15 18:43:43.804919 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.804650 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3ef401653b5b0822cfe3e5c1d154b08535d226231a55c9606ea0e5069c3a32a" Jul 15 18:43:43.836289 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.836247 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657"] Jul 15 18:43:43.842799 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.842758 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-d6657"] Jul 15 18:43:43.997428 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:43:43.997390 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" path="/var/lib/kubelet/pods/c7983222-91ce-4991-8694-e93bc7cad5b0/volumes" Jul 15 18:44:20.132915 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.132880 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6"] Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133195 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133205 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133214 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133219 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133226 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133232 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133285 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.133388 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.133292 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:20.136278 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.136261 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:20.138977 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.138954 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:44:20.139073 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.138992 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:44:20.144874 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.144848 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6"] Jul 15 18:44:20.245515 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.245479 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0184b5bd-c193-4543-99d8-7c22fcd36998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6\" (UID: \"0184b5bd-c193-4543-99d8-7c22fcd36998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:20.346521 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.346476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0184b5bd-c193-4543-99d8-7c22fcd36998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6\" (UID: \"0184b5bd-c193-4543-99d8-7c22fcd36998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:20.347099 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.347079 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0184b5bd-c193-4543-99d8-7c22fcd36998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6\" (UID: \"0184b5bd-c193-4543-99d8-7c22fcd36998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:20.447481 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.447375 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:20.580222 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.580190 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6"] Jul 15 18:44:20.583352 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:44:20.583326 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0184b5bd_c193_4543_99d8_7c22fcd36998.slice/crio-77cf049b52dcffb13effb105a4ba1f302dec7314e4e30e75ffda32238ad58bb9 WatchSource:0}: Error finding container 77cf049b52dcffb13effb105a4ba1f302dec7314e4e30e75ffda32238ad58bb9: Status 404 returned error can't find the container with id 77cf049b52dcffb13effb105a4ba1f302dec7314e4e30e75ffda32238ad58bb9 Jul 15 18:44:20.964674 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.964641 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" event={"ID":"0184b5bd-c193-4543-99d8-7c22fcd36998","Type":"ContainerStarted","Data":"b2fcb89f9c04de1f6e69ec3527c5b87918e0c4b3b5d36d675ab6c69ee6eaa8db"} Jul 15 18:44:20.964880 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.964691 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" event={"ID":"0184b5bd-c193-4543-99d8-7c22fcd36998","Type":"ContainerStarted","Data":"77cf049b52dcffb13effb105a4ba1f302dec7314e4e30e75ffda32238ad58bb9"} Jul 15 18:44:20.979920 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:20.979876 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" podStartSLOduration=0.97986207 podStartE2EDuration="979.86207ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:44:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:44:20.978542246 +0000 UTC m=+1641.588355434" watchObservedRunningTime="2026-07-15 18:44:20.97986207 +0000 UTC m=+1641.589675256" Jul 15 18:44:21.970173 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:21.970136 2557 generic.go:356] "Generic (PLEG): container finished" podID="0184b5bd-c193-4543-99d8-7c22fcd36998" containerID="b2fcb89f9c04de1f6e69ec3527c5b87918e0c4b3b5d36d675ab6c69ee6eaa8db" exitCode=0 Jul 15 18:44:21.970537 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:21.970224 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" event={"ID":"0184b5bd-c193-4543-99d8-7c22fcd36998","Type":"ContainerDied","Data":"b2fcb89f9c04de1f6e69ec3527c5b87918e0c4b3b5d36d675ab6c69ee6eaa8db"} Jul 15 18:44:23.104079 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.104055 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:23.269803 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.269686 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0184b5bd-c193-4543-99d8-7c22fcd36998-maas-api-service-ca\") pod \"0184b5bd-c193-4543-99d8-7c22fcd36998\" (UID: \"0184b5bd-c193-4543-99d8-7c22fcd36998\") " Jul 15 18:44:23.270083 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.270062 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0184b5bd-c193-4543-99d8-7c22fcd36998-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0184b5bd-c193-4543-99d8-7c22fcd36998" (UID: "0184b5bd-c193-4543-99d8-7c22fcd36998"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:44:23.370671 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.370620 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0184b5bd-c193-4543-99d8-7c22fcd36998-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:44:23.980007 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.979981 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" Jul 15 18:44:23.980007 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.980004 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6" event={"ID":"0184b5bd-c193-4543-99d8-7c22fcd36998","Type":"ContainerDied","Data":"77cf049b52dcffb13effb105a4ba1f302dec7314e4e30e75ffda32238ad58bb9"} Jul 15 18:44:23.980192 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:23.980044 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77cf049b52dcffb13effb105a4ba1f302dec7314e4e30e75ffda32238ad58bb9" Jul 15 18:44:51.907984 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.907948 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n"] Jul 15 18:44:51.908434 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.908333 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0184b5bd-c193-4543-99d8-7c22fcd36998" containerName="revoke-keys" Jul 15 18:44:51.908434 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.908348 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0184b5bd-c193-4543-99d8-7c22fcd36998" containerName="revoke-keys" Jul 15 18:44:51.908434 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.908409 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7983222-91ce-4991-8694-e93bc7cad5b0" containerName="revoke-keys" Jul 15 18:44:51.908434 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.908419 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0184b5bd-c193-4543-99d8-7c22fcd36998" containerName="revoke-keys" Jul 15 18:44:51.911459 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.911442 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:44:51.914139 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.914114 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:44:51.914139 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.914127 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:44:51.918601 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:51.918574 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n"] Jul 15 18:44:52.030896 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:52.030852 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bbc397-0782-4913-a70d-e3d1c38ed57b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-827ccb59-t475n\" (UID: \"c8bbc397-0782-4913-a70d-e3d1c38ed57b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:44:52.132065 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:52.132025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bbc397-0782-4913-a70d-e3d1c38ed57b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-827ccb59-t475n\" (UID: \"c8bbc397-0782-4913-a70d-e3d1c38ed57b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:44:52.132660 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:52.132636 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bbc397-0782-4913-a70d-e3d1c38ed57b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-827ccb59-t475n\" (UID: \"c8bbc397-0782-4913-a70d-e3d1c38ed57b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:44:52.223264 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:52.223180 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:44:52.564786 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:52.564686 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n"] Jul 15 18:44:52.568202 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:44:52.568168 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8bbc397_0782_4913_a70d_e3d1c38ed57b.slice/crio-89cd7d83a45ccc3e743df605fb8597dc4f4326f37dd8397f7b2420451565c47b WatchSource:0}: Error finding container 89cd7d83a45ccc3e743df605fb8597dc4f4326f37dd8397f7b2420451565c47b: Status 404 returned error can't find the container with id 89cd7d83a45ccc3e743df605fb8597dc4f4326f37dd8397f7b2420451565c47b Jul 15 18:44:53.115809 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:53.115754 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" event={"ID":"c8bbc397-0782-4913-a70d-e3d1c38ed57b","Type":"ContainerStarted","Data":"21cd9c18f513499a3b3c5abddd2863885f6dbb65a0732379debb2478bc3a624c"} Jul 15 18:44:53.116199 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:53.115828 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" event={"ID":"c8bbc397-0782-4913-a70d-e3d1c38ed57b","Type":"ContainerStarted","Data":"89cd7d83a45ccc3e743df605fb8597dc4f4326f37dd8397f7b2420451565c47b"} Jul 15 18:44:53.135169 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:53.135102 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" podStartSLOduration=2.135077206 podStartE2EDuration="2.135077206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:44:53.135043425 +0000 UTC m=+1673.744856614" watchObservedRunningTime="2026-07-15 18:44:53.135077206 +0000 UTC m=+1673.744890396" Jul 15 18:44:54.121828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:54.121787 2557 generic.go:356] "Generic (PLEG): container finished" podID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerID="21cd9c18f513499a3b3c5abddd2863885f6dbb65a0732379debb2478bc3a624c" exitCode=7 Jul 15 18:44:54.121828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:54.121809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" event={"ID":"c8bbc397-0782-4913-a70d-e3d1c38ed57b","Type":"ContainerDied","Data":"21cd9c18f513499a3b3c5abddd2863885f6dbb65a0732379debb2478bc3a624c"} Jul 15 18:44:54.122252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:54.122006 2557 scope.go:117] "RemoveContainer" containerID="21cd9c18f513499a3b3c5abddd2863885f6dbb65a0732379debb2478bc3a624c" Jul 15 18:44:55.129544 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:55.129511 2557 generic.go:356] "Generic (PLEG): container finished" podID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerID="94526922fd8792faa9a9002e3cac776ab9648d5219ec0f445c553c75e5295902" exitCode=7 Jul 15 18:44:55.129955 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:55.129550 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" event={"ID":"c8bbc397-0782-4913-a70d-e3d1c38ed57b","Type":"ContainerDied","Data":"94526922fd8792faa9a9002e3cac776ab9648d5219ec0f445c553c75e5295902"} Jul 15 18:44:55.129955 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:55.129596 2557 scope.go:117] "RemoveContainer" containerID="21cd9c18f513499a3b3c5abddd2863885f6dbb65a0732379debb2478bc3a624c" Jul 15 18:44:55.129955 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:55.129758 2557 scope.go:117] "RemoveContainer" containerID="94526922fd8792faa9a9002e3cac776ab9648d5219ec0f445c553c75e5295902" Jul 15 18:44:55.130090 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:44:55.130018 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-827ccb59-t475n_odh-ai-gateway-infra(c8bbc397-0782-4913-a70d-e3d1c38ed57b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" Jul 15 18:44:56.135706 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:44:56.135672 2557 scope.go:117] "RemoveContainer" containerID="94526922fd8792faa9a9002e3cac776ab9648d5219ec0f445c553c75e5295902" Jul 15 18:44:56.136186 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:44:56.135963 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-827ccb59-t475n_odh-ai-gateway-infra(c8bbc397-0782-4913-a70d-e3d1c38ed57b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" Jul 15 18:45:00.130718 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.130681 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb"] Jul 15 18:45:00.135422 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.135397 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d"] Jul 15 18:45:00.135563 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.135515 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" Jul 15 18:45:00.139002 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.138977 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" Jul 15 18:45:00.140655 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.140631 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d"] Jul 15 18:45:00.147694 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.147669 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb"] Jul 15 18:45:00.150445 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.150418 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" Jul 15 18:45:00.162493 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.162468 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" Jul 15 18:45:00.343392 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.343259 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d"] Jul 15 18:45:00.345540 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:45:00.345516 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedb3d60_8510_46ef_8283_4debd86915dd.slice/crio-c3a2c30abc1c68b16e8b24ecef001d5a5c3591e2c15fa2df213724de65c166c4 WatchSource:0}: Error finding container c3a2c30abc1c68b16e8b24ecef001d5a5c3591e2c15fa2df213724de65c166c4: Status 404 returned error can't find the container with id c3a2c30abc1c68b16e8b24ecef001d5a5c3591e2c15fa2df213724de65c166c4 Jul 15 18:45:00.526410 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:00.526382 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb"] Jul 15 18:45:00.528688 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:45:00.528652 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd635adc8_0bfe_4cbf_ab6a_8aa203003594.slice/crio-92336b36a30d07393374a7db5ba32304434c0e812f5e424d1df9773a00debe35 WatchSource:0}: Error finding container 92336b36a30d07393374a7db5ba32304434c0e812f5e424d1df9773a00debe35: Status 404 returned error can't find the container with id 92336b36a30d07393374a7db5ba32304434c0e812f5e424d1df9773a00debe35 Jul 15 18:45:01.164435 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:01.164398 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" event={"ID":"dedb3d60-8510-46ef-8283-4debd86915dd","Type":"ContainerStarted","Data":"9771517a89d58d51686cff2efbaf45a321195bc6382fd8bf7c31be8481442b38"} Jul 15 18:45:01.164863 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:01.164462 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" event={"ID":"dedb3d60-8510-46ef-8283-4debd86915dd","Type":"ContainerStarted","Data":"c3a2c30abc1c68b16e8b24ecef001d5a5c3591e2c15fa2df213724de65c166c4"} Jul 15 18:45:01.165999 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:01.165975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" event={"ID":"d635adc8-0bfe-4cbf-ab6a-8aa203003594","Type":"ContainerStarted","Data":"1026bb99d87a707517a7b2d7570aa3264ae6fcf069bb7fb5a4b4455799e9e95b"} Jul 15 18:45:01.166096 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:01.166016 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" event={"ID":"d635adc8-0bfe-4cbf-ab6a-8aa203003594","Type":"ContainerStarted","Data":"92336b36a30d07393374a7db5ba32304434c0e812f5e424d1df9773a00debe35"} Jul 15 18:45:01.180809 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:01.180742 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" podStartSLOduration=1.180724116 podStartE2EDuration="1.180724116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:45:01.1797034 +0000 UTC m=+1681.789516589" watchObservedRunningTime="2026-07-15 18:45:01.180724116 +0000 UTC m=+1681.790537286" Jul 15 18:45:02.172195 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:02.172161 2557 generic.go:356] "Generic (PLEG): container finished" podID="dedb3d60-8510-46ef-8283-4debd86915dd" containerID="9771517a89d58d51686cff2efbaf45a321195bc6382fd8bf7c31be8481442b38" exitCode=0 Jul 15 18:45:02.172618 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:02.172246 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" event={"ID":"dedb3d60-8510-46ef-8283-4debd86915dd","Type":"ContainerDied","Data":"9771517a89d58d51686cff2efbaf45a321195bc6382fd8bf7c31be8481442b38"} Jul 15 18:45:02.174125 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:02.174094 2557 generic.go:356] "Generic (PLEG): container finished" podID="d635adc8-0bfe-4cbf-ab6a-8aa203003594" containerID="1026bb99d87a707517a7b2d7570aa3264ae6fcf069bb7fb5a4b4455799e9e95b" exitCode=0 Jul 15 18:45:02.174249 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:02.174145 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" event={"ID":"d635adc8-0bfe-4cbf-ab6a-8aa203003594","Type":"ContainerDied","Data":"1026bb99d87a707517a7b2d7570aa3264ae6fcf069bb7fb5a4b4455799e9e95b"} Jul 15 18:45:02.188123 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:02.188074 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" podStartSLOduration=2.188057019 podStartE2EDuration="2.188057019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:45:01.194475965 +0000 UTC m=+1681.804289153" watchObservedRunningTime="2026-07-15 18:45:02.188057019 +0000 UTC m=+1682.797870207" Jul 15 18:45:03.336624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:03.336600 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" Jul 15 18:45:03.340168 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:03.340146 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" Jul 15 18:45:04.184642 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:04.184612 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" Jul 15 18:45:04.184859 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:04.184612 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735685-fcp5d" event={"ID":"dedb3d60-8510-46ef-8283-4debd86915dd","Type":"ContainerDied","Data":"c3a2c30abc1c68b16e8b24ecef001d5a5c3591e2c15fa2df213724de65c166c4"} Jul 15 18:45:04.184859 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:04.184758 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3a2c30abc1c68b16e8b24ecef001d5a5c3591e2c15fa2df213724de65c166c4" Jul 15 18:45:04.185944 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:04.185926 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" Jul 15 18:45:04.185944 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:04.185927 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb" event={"ID":"d635adc8-0bfe-4cbf-ab6a-8aa203003594","Type":"ContainerDied","Data":"92336b36a30d07393374a7db5ba32304434c0e812f5e424d1df9773a00debe35"} Jul 15 18:45:04.186074 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:04.185969 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92336b36a30d07393374a7db5ba32304434c0e812f5e424d1df9773a00debe35" Jul 15 18:45:09.993999 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:09.993972 2557 scope.go:117] "RemoveContainer" containerID="94526922fd8792faa9a9002e3cac776ab9648d5219ec0f445c553c75e5295902" Jul 15 18:45:11.219183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:11.219149 2557 generic.go:356] "Generic (PLEG): container finished" podID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerID="21b223409d42573efeacb2134a590c819f2fbc543a9b52b4f88df8f3c1f050cf" exitCode=0 Jul 15 18:45:11.219594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:11.219221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" event={"ID":"c8bbc397-0782-4913-a70d-e3d1c38ed57b","Type":"ContainerDied","Data":"21b223409d42573efeacb2134a590c819f2fbc543a9b52b4f88df8f3c1f050cf"} Jul 15 18:45:11.219594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:11.219313 2557 scope.go:117] "RemoveContainer" containerID="94526922fd8792faa9a9002e3cac776ab9648d5219ec0f445c553c75e5295902" Jul 15 18:45:12.248215 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:12.248181 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n"] Jul 15 18:45:12.358686 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:12.358659 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:45:12.522909 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:12.522822 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bbc397-0782-4913-a70d-e3d1c38ed57b-maas-api-service-ca\") pod \"c8bbc397-0782-4913-a70d-e3d1c38ed57b\" (UID: \"c8bbc397-0782-4913-a70d-e3d1c38ed57b\") " Jul 15 18:45:12.523194 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:12.523168 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8bbc397-0782-4913-a70d-e3d1c38ed57b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8bbc397-0782-4913-a70d-e3d1c38ed57b" (UID: "c8bbc397-0782-4913-a70d-e3d1c38ed57b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:45:12.624269 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:12.624231 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bbc397-0782-4913-a70d-e3d1c38ed57b-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:45:13.232125 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:13.232096 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" Jul 15 18:45:13.232294 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:13.232130 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n" event={"ID":"c8bbc397-0782-4913-a70d-e3d1c38ed57b","Type":"ContainerDied","Data":"89cd7d83a45ccc3e743df605fb8597dc4f4326f37dd8397f7b2420451565c47b"} Jul 15 18:45:13.232294 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:13.232185 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89cd7d83a45ccc3e743df605fb8597dc4f4326f37dd8397f7b2420451565c47b" Jul 15 18:45:13.285594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:13.285558 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n"] Jul 15 18:45:13.295191 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:13.295160 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-t475n"] Jul 15 18:45:13.998249 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:13.998215 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" path="/var/lib/kubelet/pods/c8bbc397-0782-4913-a70d-e3d1c38ed57b/volumes" Jul 15 18:45:51.955247 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955213 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6"] Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955521 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955534 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955554 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d635adc8-0bfe-4cbf-ab6a-8aa203003594" containerName="cleanup" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955560 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d635adc8-0bfe-4cbf-ab6a-8aa203003594" containerName="cleanup" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955568 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955574 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955584 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dedb3d60-8510-46ef-8283-4debd86915dd" containerName="cleanup" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955590 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dedb3d60-8510-46ef-8283-4debd86915dd" containerName="cleanup" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955637 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d635adc8-0bfe-4cbf-ab6a-8aa203003594" containerName="cleanup" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955645 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dedb3d60-8510-46ef-8283-4debd86915dd" containerName="cleanup" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955651 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955657 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.955687 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.955662 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:45:51.958674 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.958656 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:51.961154 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.961133 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:45:51.961242 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.961168 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:45:51.964680 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:51.964655 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6"] Jul 15 18:45:52.073172 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:52.073135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c38348d3-7fbf-49de-ac17-c1e7e76c9f8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6\" (UID: \"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:52.174313 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:52.174265 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c38348d3-7fbf-49de-ac17-c1e7e76c9f8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6\" (UID: \"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:52.174982 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:52.174961 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c38348d3-7fbf-49de-ac17-c1e7e76c9f8c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6\" (UID: \"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:52.271276 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:52.271189 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:52.406837 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:52.406813 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6"] Jul 15 18:45:52.408939 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:45:52.408902 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc38348d3_7fbf_49de_ac17_c1e7e76c9f8c.slice/crio-a9d5e67343cd1d45af99a83847fded3dd5e72fc49f2aa33d4c050a62f9843ab3 WatchSource:0}: Error finding container a9d5e67343cd1d45af99a83847fded3dd5e72fc49f2aa33d4c050a62f9843ab3: Status 404 returned error can't find the container with id a9d5e67343cd1d45af99a83847fded3dd5e72fc49f2aa33d4c050a62f9843ab3 Jul 15 18:45:53.412628 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:53.412547 2557 generic.go:356] "Generic (PLEG): container finished" podID="c38348d3-7fbf-49de-ac17-c1e7e76c9f8c" containerID="5a35c13db8f6d446fe74e1f413e116103e0c3a43850a516f885afae277cedf36" exitCode=0 Jul 15 18:45:53.413004 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:53.412631 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" event={"ID":"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c","Type":"ContainerDied","Data":"5a35c13db8f6d446fe74e1f413e116103e0c3a43850a516f885afae277cedf36"} Jul 15 18:45:53.413004 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:53.412682 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" event={"ID":"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c","Type":"ContainerStarted","Data":"a9d5e67343cd1d45af99a83847fded3dd5e72fc49f2aa33d4c050a62f9843ab3"} Jul 15 18:45:54.552713 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:54.552683 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:54.696115 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:54.696017 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c38348d3-7fbf-49de-ac17-c1e7e76c9f8c-maas-api-service-ca\") pod \"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c\" (UID: \"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c\") " Jul 15 18:45:54.696392 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:54.696365 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c38348d3-7fbf-49de-ac17-c1e7e76c9f8c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c38348d3-7fbf-49de-ac17-c1e7e76c9f8c" (UID: "c38348d3-7fbf-49de-ac17-c1e7e76c9f8c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:45:54.797510 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:54.797474 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c38348d3-7fbf-49de-ac17-c1e7e76c9f8c-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:45:55.424076 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:55.424046 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" Jul 15 18:45:55.424240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:55.424078 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6" event={"ID":"c38348d3-7fbf-49de-ac17-c1e7e76c9f8c","Type":"ContainerDied","Data":"a9d5e67343cd1d45af99a83847fded3dd5e72fc49f2aa33d4c050a62f9843ab3"} Jul 15 18:45:55.424240 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:55.424129 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9d5e67343cd1d45af99a83847fded3dd5e72fc49f2aa33d4c050a62f9843ab3" Jul 15 18:45:57.538201 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:57.538166 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb"] Jul 15 18:45:57.541811 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:57.541785 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-827ccb59-29735685-2tnnb"] Jul 15 18:45:57.997313 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:45:57.997278 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d635adc8-0bfe-4cbf-ab6a-8aa203003594" path="/var/lib/kubelet/pods/d635adc8-0bfe-4cbf-ab6a-8aa203003594/volumes" Jul 15 18:46:10.222705 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.222664 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p"] Jul 15 18:46:10.223245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.223012 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c38348d3-7fbf-49de-ac17-c1e7e76c9f8c" containerName="revoke-keys" Jul 15 18:46:10.223245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.223026 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c38348d3-7fbf-49de-ac17-c1e7e76c9f8c" containerName="revoke-keys" Jul 15 18:46:10.223245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.223037 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:46:10.223245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.223042 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8bbc397-0782-4913-a70d-e3d1c38ed57b" containerName="revoke-keys" Jul 15 18:46:10.223245 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.223100 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c38348d3-7fbf-49de-ac17-c1e7e76c9f8c" containerName="revoke-keys" Jul 15 18:46:10.225969 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.225952 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:10.228314 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.228294 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:46:10.228416 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.228319 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:46:10.232601 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.232573 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p"] Jul 15 18:46:10.323445 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.323410 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09e63238-70c6-432a-ba8b-375bd4a9c175-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cfe32069-5699p\" (UID: \"09e63238-70c6-432a-ba8b-375bd4a9c175\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:10.424682 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.424644 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09e63238-70c6-432a-ba8b-375bd4a9c175-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cfe32069-5699p\" (UID: \"09e63238-70c6-432a-ba8b-375bd4a9c175\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:10.425275 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.425257 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09e63238-70c6-432a-ba8b-375bd4a9c175-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cfe32069-5699p\" (UID: \"09e63238-70c6-432a-ba8b-375bd4a9c175\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:10.537977 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.537872 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:10.673149 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:10.673122 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p"] Jul 15 18:46:10.675310 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:46:10.675280 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e63238_70c6_432a_ba8b_375bd4a9c175.slice/crio-e49db4665e11ff133fe31dab80ebe4770883c6e0865c2da1a7e9d43775f27e75 WatchSource:0}: Error finding container e49db4665e11ff133fe31dab80ebe4770883c6e0865c2da1a7e9d43775f27e75: Status 404 returned error can't find the container with id e49db4665e11ff133fe31dab80ebe4770883c6e0865c2da1a7e9d43775f27e75 Jul 15 18:46:11.496061 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:11.496028 2557 generic.go:356] "Generic (PLEG): container finished" podID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerID="388da48b182d69ea25e2a759f22649fba19e3ff39cecc05d8806f217e8e6b516" exitCode=7 Jul 15 18:46:11.496420 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:11.496115 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" event={"ID":"09e63238-70c6-432a-ba8b-375bd4a9c175","Type":"ContainerDied","Data":"388da48b182d69ea25e2a759f22649fba19e3ff39cecc05d8806f217e8e6b516"} Jul 15 18:46:11.496420 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:11.496169 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" event={"ID":"09e63238-70c6-432a-ba8b-375bd4a9c175","Type":"ContainerStarted","Data":"e49db4665e11ff133fe31dab80ebe4770883c6e0865c2da1a7e9d43775f27e75"} Jul 15 18:46:11.496420 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:11.496259 2557 scope.go:117] "RemoveContainer" containerID="388da48b182d69ea25e2a759f22649fba19e3ff39cecc05d8806f217e8e6b516" Jul 15 18:46:12.503028 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:12.502929 2557 generic.go:356] "Generic (PLEG): container finished" podID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerID="7f8f91fe628eeb1139e6e7d8aa78c26672a42f5973315ba46de08bf6bafa632a" exitCode=7 Jul 15 18:46:12.503028 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:12.502993 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" event={"ID":"09e63238-70c6-432a-ba8b-375bd4a9c175","Type":"ContainerDied","Data":"7f8f91fe628eeb1139e6e7d8aa78c26672a42f5973315ba46de08bf6bafa632a"} Jul 15 18:46:12.503489 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:12.503047 2557 scope.go:117] "RemoveContainer" containerID="388da48b182d69ea25e2a759f22649fba19e3ff39cecc05d8806f217e8e6b516" Jul 15 18:46:12.503489 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:12.503231 2557 scope.go:117] "RemoveContainer" containerID="7f8f91fe628eeb1139e6e7d8aa78c26672a42f5973315ba46de08bf6bafa632a" Jul 15 18:46:12.503489 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:46:12.503458 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-cfe32069-5699p_odh-ai-gateway-infra(09e63238-70c6-432a-ba8b-375bd4a9c175)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" Jul 15 18:46:13.510205 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:13.510179 2557 scope.go:117] "RemoveContainer" containerID="7f8f91fe628eeb1139e6e7d8aa78c26672a42f5973315ba46de08bf6bafa632a" Jul 15 18:46:13.510575 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:46:13.510369 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-cfe32069-5699p_odh-ai-gateway-infra(09e63238-70c6-432a-ba8b-375bd4a9c175)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" Jul 15 18:46:28.992673 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:28.992642 2557 scope.go:117] "RemoveContainer" containerID="7f8f91fe628eeb1139e6e7d8aa78c26672a42f5973315ba46de08bf6bafa632a" Jul 15 18:46:29.588629 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:29.588592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" event={"ID":"09e63238-70c6-432a-ba8b-375bd4a9c175","Type":"ContainerStarted","Data":"365e105d6bde2a725ae2f217b31645dad528d2942727b989c481e373d25b87c0"} Jul 15 18:46:29.605145 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:29.605082 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" podStartSLOduration=19.605061018 podStartE2EDuration="19.605061018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:46:29.60368942 +0000 UTC m=+1770.213502627" watchObservedRunningTime="2026-07-15 18:46:29.605061018 +0000 UTC m=+1770.214874207" Jul 15 18:46:30.595225 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:30.595195 2557 generic.go:356] "Generic (PLEG): container finished" podID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerID="365e105d6bde2a725ae2f217b31645dad528d2942727b989c481e373d25b87c0" exitCode=0 Jul 15 18:46:30.595617 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:30.595263 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" event={"ID":"09e63238-70c6-432a-ba8b-375bd4a9c175","Type":"ContainerDied","Data":"365e105d6bde2a725ae2f217b31645dad528d2942727b989c481e373d25b87c0"} Jul 15 18:46:30.595617 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:30.595325 2557 scope.go:117] "RemoveContainer" containerID="7f8f91fe628eeb1139e6e7d8aa78c26672a42f5973315ba46de08bf6bafa632a" Jul 15 18:46:30.622414 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:30.622382 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p"] Jul 15 18:46:31.735585 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:31.735562 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:31.812334 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:31.812299 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09e63238-70c6-432a-ba8b-375bd4a9c175-maas-api-service-ca\") pod \"09e63238-70c6-432a-ba8b-375bd4a9c175\" (UID: \"09e63238-70c6-432a-ba8b-375bd4a9c175\") " Jul 15 18:46:31.812710 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:31.812673 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09e63238-70c6-432a-ba8b-375bd4a9c175-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09e63238-70c6-432a-ba8b-375bd4a9c175" (UID: "09e63238-70c6-432a-ba8b-375bd4a9c175"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:46:31.913879 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:31.913789 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09e63238-70c6-432a-ba8b-375bd4a9c175-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:46:32.607018 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:32.606989 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" Jul 15 18:46:32.607018 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:32.607014 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p" event={"ID":"09e63238-70c6-432a-ba8b-375bd4a9c175","Type":"ContainerDied","Data":"e49db4665e11ff133fe31dab80ebe4770883c6e0865c2da1a7e9d43775f27e75"} Jul 15 18:46:32.607344 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:32.607060 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e49db4665e11ff133fe31dab80ebe4770883c6e0865c2da1a7e9d43775f27e75" Jul 15 18:46:32.635270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:32.635219 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p"] Jul 15 18:46:32.637252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:32.637226 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-5699p"] Jul 15 18:46:33.997485 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:33.997453 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" path="/var/lib/kubelet/pods/09e63238-70c6-432a-ba8b-375bd4a9c175/volumes" Jul 15 18:46:59.949752 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:59.949649 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:46:59.956500 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:59.951686 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:46:59.956500 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:59.955615 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:46:59.957877 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:46:59.957858 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:47:10.274311 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274278 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw"] Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274585 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274595 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274604 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274610 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274668 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274675 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.274712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.274683 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:10.277659 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.277643 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:10.280170 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.280149 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:47:10.280270 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.280192 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:47:10.285016 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.284993 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw"] Jul 15 18:47:10.343336 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.343300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/412badb1-0609-4de3-82c6-423d55cfe749-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw\" (UID: \"412badb1-0609-4de3-82c6-423d55cfe749\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:10.444241 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.444203 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/412badb1-0609-4de3-82c6-423d55cfe749-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw\" (UID: \"412badb1-0609-4de3-82c6-423d55cfe749\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:10.444877 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.444859 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/412badb1-0609-4de3-82c6-423d55cfe749-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw\" (UID: \"412badb1-0609-4de3-82c6-423d55cfe749\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:10.589179 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.589136 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:10.715699 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.715667 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw"] Jul 15 18:47:10.718607 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:47:10.718578 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412badb1_0609_4de3_82c6_423d55cfe749.slice/crio-4e3aeceec32d62fcc71fb49793b16af080efd8aa9750be185301991044ca2df9 WatchSource:0}: Error finding container 4e3aeceec32d62fcc71fb49793b16af080efd8aa9750be185301991044ca2df9: Status 404 returned error can't find the container with id 4e3aeceec32d62fcc71fb49793b16af080efd8aa9750be185301991044ca2df9 Jul 15 18:47:10.783161 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:10.783127 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" event={"ID":"412badb1-0609-4de3-82c6-423d55cfe749","Type":"ContainerStarted","Data":"4e3aeceec32d62fcc71fb49793b16af080efd8aa9750be185301991044ca2df9"} Jul 15 18:47:11.789710 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:11.789627 2557 generic.go:356] "Generic (PLEG): container finished" podID="412badb1-0609-4de3-82c6-423d55cfe749" containerID="ae42f84f63fa0afce66f800c85c819b1ded8bdaaa61f60474824599f8c6efa22" exitCode=0 Jul 15 18:47:11.790077 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:11.789717 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" event={"ID":"412badb1-0609-4de3-82c6-423d55cfe749","Type":"ContainerDied","Data":"ae42f84f63fa0afce66f800c85c819b1ded8bdaaa61f60474824599f8c6efa22"} Jul 15 18:47:12.926349 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:12.926319 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:12.968322 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:12.968292 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/412badb1-0609-4de3-82c6-423d55cfe749-maas-api-service-ca\") pod \"412badb1-0609-4de3-82c6-423d55cfe749\" (UID: \"412badb1-0609-4de3-82c6-423d55cfe749\") " Jul 15 18:47:12.968642 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:12.968620 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/412badb1-0609-4de3-82c6-423d55cfe749-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "412badb1-0609-4de3-82c6-423d55cfe749" (UID: "412badb1-0609-4de3-82c6-423d55cfe749"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:47:13.069429 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:13.069329 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/412badb1-0609-4de3-82c6-423d55cfe749-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:47:13.800117 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:13.800088 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" Jul 15 18:47:13.800286 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:13.800114 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw" event={"ID":"412badb1-0609-4de3-82c6-423d55cfe749","Type":"ContainerDied","Data":"4e3aeceec32d62fcc71fb49793b16af080efd8aa9750be185301991044ca2df9"} Jul 15 18:47:13.800286 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:13.800164 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e3aeceec32d62fcc71fb49793b16af080efd8aa9750be185301991044ca2df9" Jul 15 18:47:42.227752 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.227713 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq"] Jul 15 18:47:42.228212 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.228048 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="412badb1-0609-4de3-82c6-423d55cfe749" containerName="revoke-keys" Jul 15 18:47:42.228212 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.228059 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="412badb1-0609-4de3-82c6-423d55cfe749" containerName="revoke-keys" Jul 15 18:47:42.228212 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.228072 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:42.228212 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.228078 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09e63238-70c6-432a-ba8b-375bd4a9c175" containerName="revoke-keys" Jul 15 18:47:42.228212 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.228133 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="412badb1-0609-4de3-82c6-423d55cfe749" containerName="revoke-keys" Jul 15 18:47:42.231044 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.231026 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:47:42.233437 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.233413 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:47:42.233544 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.233469 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:47:42.237128 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.237102 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq"] Jul 15 18:47:42.327572 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.327518 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq\" (UID: \"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:47:42.429094 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.429053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq\" (UID: \"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:47:42.429696 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.429676 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq\" (UID: \"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:47:42.542917 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.542821 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:47:42.673552 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.673528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq"] Jul 15 18:47:42.675591 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:47:42.675562 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c8c8e5_6ee5_4c42_97d0_d7a49316ecab.slice/crio-d88e1f73479bf294c8db7bf03879366ecd755f4d474ee195eb23ad5e9210c32d WatchSource:0}: Error finding container d88e1f73479bf294c8db7bf03879366ecd755f4d474ee195eb23ad5e9210c32d: Status 404 returned error can't find the container with id d88e1f73479bf294c8db7bf03879366ecd755f4d474ee195eb23ad5e9210c32d Jul 15 18:47:42.945813 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:42.945757 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" event={"ID":"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab","Type":"ContainerStarted","Data":"d88e1f73479bf294c8db7bf03879366ecd755f4d474ee195eb23ad5e9210c32d"} Jul 15 18:47:43.951593 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:43.951556 2557 generic.go:356] "Generic (PLEG): container finished" podID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerID="c6e09e627e05c2e74d36869d6a44f8c7d7f590b7c3b07ed3b6b29d1e85d8a0ee" exitCode=7 Jul 15 18:47:43.952036 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:43.951639 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" event={"ID":"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab","Type":"ContainerDied","Data":"c6e09e627e05c2e74d36869d6a44f8c7d7f590b7c3b07ed3b6b29d1e85d8a0ee"} Jul 15 18:47:43.952036 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:43.951857 2557 scope.go:117] "RemoveContainer" containerID="c6e09e627e05c2e74d36869d6a44f8c7d7f590b7c3b07ed3b6b29d1e85d8a0ee" Jul 15 18:47:44.958349 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:44.958315 2557 generic.go:356] "Generic (PLEG): container finished" podID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerID="dba49edebab357a5eb18250761c5be17074054d286580b29622b99f986c72fed" exitCode=7 Jul 15 18:47:44.958755 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:44.958375 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" event={"ID":"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab","Type":"ContainerDied","Data":"dba49edebab357a5eb18250761c5be17074054d286580b29622b99f986c72fed"} Jul 15 18:47:44.958755 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:44.958421 2557 scope.go:117] "RemoveContainer" containerID="c6e09e627e05c2e74d36869d6a44f8c7d7f590b7c3b07ed3b6b29d1e85d8a0ee" Jul 15 18:47:44.958755 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:44.958611 2557 scope.go:117] "RemoveContainer" containerID="dba49edebab357a5eb18250761c5be17074054d286580b29622b99f986c72fed" Jul 15 18:47:44.958893 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:47:44.958854 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq_odh-ai-gateway-infra(12c8c8e5-6ee5-4c42-97d0-d7a49316ecab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" Jul 15 18:47:45.964623 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:45.964595 2557 scope.go:117] "RemoveContainer" containerID="dba49edebab357a5eb18250761c5be17074054d286580b29622b99f986c72fed" Jul 15 18:47:45.965031 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:47:45.964833 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq_odh-ai-gateway-infra(12c8c8e5-6ee5-4c42-97d0-d7a49316ecab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" Jul 15 18:47:57.992608 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:57.992572 2557 scope.go:117] "RemoveContainer" containerID="dba49edebab357a5eb18250761c5be17074054d286580b29622b99f986c72fed" Jul 15 18:47:59.028288 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:59.028254 2557 generic.go:356] "Generic (PLEG): container finished" podID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerID="e62ce3053eb93e16f820e0872e8d5684f32b9c11f968463306c0fc34d3576274" exitCode=0 Jul 15 18:47:59.028659 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:59.028301 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" event={"ID":"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab","Type":"ContainerDied","Data":"e62ce3053eb93e16f820e0872e8d5684f32b9c11f968463306c0fc34d3576274"} Jul 15 18:47:59.028659 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:47:59.028352 2557 scope.go:117] "RemoveContainer" containerID="dba49edebab357a5eb18250761c5be17074054d286580b29622b99f986c72fed" Jul 15 18:48:00.058494 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:00.058459 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq"] Jul 15 18:48:00.168823 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:00.168793 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:48:00.285277 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:00.285183 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab-maas-api-service-ca\") pod \"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab\" (UID: \"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab\") " Jul 15 18:48:00.285586 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:00.285560 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" (UID: "12c8c8e5-6ee5-4c42-97d0-d7a49316ecab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:48:00.386004 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:00.385967 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:48:01.039588 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:01.039554 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" event={"ID":"12c8c8e5-6ee5-4c42-97d0-d7a49316ecab","Type":"ContainerDied","Data":"d88e1f73479bf294c8db7bf03879366ecd755f4d474ee195eb23ad5e9210c32d"} Jul 15 18:48:01.039782 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:01.039606 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d88e1f73479bf294c8db7bf03879366ecd755f4d474ee195eb23ad5e9210c32d" Jul 15 18:48:01.039782 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:01.039564 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq" Jul 15 18:48:01.071413 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:01.071380 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq"] Jul 15 18:48:01.076909 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:01.076881 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-l4jfq"] Jul 15 18:48:01.997276 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:01.997244 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" path="/var/lib/kubelet/pods/12c8c8e5-6ee5-4c42-97d0-d7a49316ecab/volumes" Jul 15 18:48:32.271354 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271317 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r"] Jul 15 18:48:32.271828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271631 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:48:32.271828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271642 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:48:32.271828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271656 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:48:32.271828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271662 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:48:32.271828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271721 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:48:32.271828 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.271732 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:48:32.274608 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.274592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:48:32.277305 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.277277 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:48:32.277305 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.277307 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:48:32.282187 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.281939 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r"] Jul 15 18:48:32.372221 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.372185 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d257934-f5cb-4f47-92f7-d29c2d9832f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r\" (UID: \"4d257934-f5cb-4f47-92f7-d29c2d9832f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:48:32.473253 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.473213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d257934-f5cb-4f47-92f7-d29c2d9832f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r\" (UID: \"4d257934-f5cb-4f47-92f7-d29c2d9832f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:48:32.473882 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.473863 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d257934-f5cb-4f47-92f7-d29c2d9832f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r\" (UID: \"4d257934-f5cb-4f47-92f7-d29c2d9832f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:48:32.585896 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.585807 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:48:32.716340 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:32.716314 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r"] Jul 15 18:48:32.718162 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:48:32.718133 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d257934_f5cb_4f47_92f7_d29c2d9832f1.slice/crio-78ea7791f19c87ef83c8cec3d695f3a2d9ee3d06439f554d67541496bec0be98 WatchSource:0}: Error finding container 78ea7791f19c87ef83c8cec3d695f3a2d9ee3d06439f554d67541496bec0be98: Status 404 returned error can't find the container with id 78ea7791f19c87ef83c8cec3d695f3a2d9ee3d06439f554d67541496bec0be98 Jul 15 18:48:33.189609 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:33.189511 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" event={"ID":"4d257934-f5cb-4f47-92f7-d29c2d9832f1","Type":"ContainerStarted","Data":"f31cf4e72f2e48d068fa25073a0b685ef33294ce9ecc0f92738c179574faffa2"} Jul 15 18:48:33.189609 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:33.189581 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" event={"ID":"4d257934-f5cb-4f47-92f7-d29c2d9832f1","Type":"ContainerStarted","Data":"78ea7791f19c87ef83c8cec3d695f3a2d9ee3d06439f554d67541496bec0be98"} Jul 15 18:48:33.206330 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:33.206276 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" podStartSLOduration=1.2062589240000001 podStartE2EDuration="1.206258924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:48:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:48:33.205103327 +0000 UTC m=+1893.814916515" watchObservedRunningTime="2026-07-15 18:48:33.206258924 +0000 UTC m=+1893.816072112" Jul 15 18:48:34.195972 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:34.195936 2557 generic.go:356] "Generic (PLEG): container finished" podID="4d257934-f5cb-4f47-92f7-d29c2d9832f1" containerID="f31cf4e72f2e48d068fa25073a0b685ef33294ce9ecc0f92738c179574faffa2" exitCode=0 Jul 15 18:48:34.196358 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:34.196019 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" event={"ID":"4d257934-f5cb-4f47-92f7-d29c2d9832f1","Type":"ContainerDied","Data":"f31cf4e72f2e48d068fa25073a0b685ef33294ce9ecc0f92738c179574faffa2"} Jul 15 18:48:35.326259 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:35.326235 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:48:35.500282 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:35.500187 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d257934-f5cb-4f47-92f7-d29c2d9832f1-maas-api-service-ca\") pod \"4d257934-f5cb-4f47-92f7-d29c2d9832f1\" (UID: \"4d257934-f5cb-4f47-92f7-d29c2d9832f1\") " Jul 15 18:48:35.500551 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:35.500527 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d257934-f5cb-4f47-92f7-d29c2d9832f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d257934-f5cb-4f47-92f7-d29c2d9832f1" (UID: "4d257934-f5cb-4f47-92f7-d29c2d9832f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:48:35.601165 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:35.601132 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d257934-f5cb-4f47-92f7-d29c2d9832f1-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:48:36.206552 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:36.206507 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" event={"ID":"4d257934-f5cb-4f47-92f7-d29c2d9832f1","Type":"ContainerDied","Data":"78ea7791f19c87ef83c8cec3d695f3a2d9ee3d06439f554d67541496bec0be98"} Jul 15 18:48:36.206741 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:36.206564 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78ea7791f19c87ef83c8cec3d695f3a2d9ee3d06439f554d67541496bec0be98" Jul 15 18:48:36.206741 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:48:36.206524 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r" Jul 15 18:50:00.007033 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:00.006952 2557 scope.go:117] "RemoveContainer" containerID="9cc9d20040bbd6dc357a65e3fb7bf367d41c7f569aee258c5d82ca56b0a0a211" Jul 15 18:50:53.382826 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.382790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms"] Jul 15 18:50:53.383251 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.383120 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:50:53.383251 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.383130 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:50:53.383251 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.383150 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d257934-f5cb-4f47-92f7-d29c2d9832f1" containerName="revoke-keys" Jul 15 18:50:53.383251 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.383155 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d257934-f5cb-4f47-92f7-d29c2d9832f1" containerName="revoke-keys" Jul 15 18:50:53.383251 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.383215 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c8c8e5-6ee5-4c42-97d0-d7a49316ecab" containerName="revoke-keys" Jul 15 18:50:53.383251 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.383222 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d257934-f5cb-4f47-92f7-d29c2d9832f1" containerName="revoke-keys" Jul 15 18:50:53.386247 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.386232 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:53.389218 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.389192 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:50:53.389427 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.389414 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:50:53.398844 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.398823 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms"] Jul 15 18:50:53.448600 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.448557 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d06ea689-ffa4-44c4-93c4-c2db0bbc5724-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms\" (UID: \"d06ea689-ffa4-44c4-93c4-c2db0bbc5724\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:53.549292 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.549249 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d06ea689-ffa4-44c4-93c4-c2db0bbc5724-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms\" (UID: \"d06ea689-ffa4-44c4-93c4-c2db0bbc5724\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:53.549887 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.549867 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d06ea689-ffa4-44c4-93c4-c2db0bbc5724-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms\" (UID: \"d06ea689-ffa4-44c4-93c4-c2db0bbc5724\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:53.695949 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.695844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:53.835890 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.835864 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms"] Jul 15 18:50:53.838124 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:50:53.838095 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06ea689_ffa4_44c4_93c4_c2db0bbc5724.slice/crio-b51d71d0c365ee88d9df7bb4d1904b9f807da0881817e887dbdc21de4ee6308e WatchSource:0}: Error finding container b51d71d0c365ee88d9df7bb4d1904b9f807da0881817e887dbdc21de4ee6308e: Status 404 returned error can't find the container with id b51d71d0c365ee88d9df7bb4d1904b9f807da0881817e887dbdc21de4ee6308e Jul 15 18:50:53.865072 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:53.865039 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" event={"ID":"d06ea689-ffa4-44c4-93c4-c2db0bbc5724","Type":"ContainerStarted","Data":"b51d71d0c365ee88d9df7bb4d1904b9f807da0881817e887dbdc21de4ee6308e"} Jul 15 18:50:54.870830 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:54.870795 2557 generic.go:356] "Generic (PLEG): container finished" podID="d06ea689-ffa4-44c4-93c4-c2db0bbc5724" containerID="9f86b21640f391082c8168598d31e13c012f8f5e225f477cafa0be02a0a14ea5" exitCode=0 Jul 15 18:50:54.871228 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:54.870887 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" event={"ID":"d06ea689-ffa4-44c4-93c4-c2db0bbc5724","Type":"ContainerDied","Data":"9f86b21640f391082c8168598d31e13c012f8f5e225f477cafa0be02a0a14ea5"} Jul 15 18:50:56.007447 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.007421 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:56.072284 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.072199 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d06ea689-ffa4-44c4-93c4-c2db0bbc5724-maas-api-service-ca\") pod \"d06ea689-ffa4-44c4-93c4-c2db0bbc5724\" (UID: \"d06ea689-ffa4-44c4-93c4-c2db0bbc5724\") " Jul 15 18:50:56.072602 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.072577 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06ea689-ffa4-44c4-93c4-c2db0bbc5724-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d06ea689-ffa4-44c4-93c4-c2db0bbc5724" (UID: "d06ea689-ffa4-44c4-93c4-c2db0bbc5724"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:50:56.072668 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.072653 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d06ea689-ffa4-44c4-93c4-c2db0bbc5724-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:50:56.881704 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.881673 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" event={"ID":"d06ea689-ffa4-44c4-93c4-c2db0bbc5724","Type":"ContainerDied","Data":"b51d71d0c365ee88d9df7bb4d1904b9f807da0881817e887dbdc21de4ee6308e"} Jul 15 18:50:56.881704 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.881697 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms" Jul 15 18:50:56.881704 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:50:56.881722 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b51d71d0c365ee88d9df7bb4d1904b9f807da0881817e887dbdc21de4ee6308e" Jul 15 18:51:59.981588 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:51:59.981482 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:51:59.984204 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:51:59.984182 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:51:59.987625 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:51:59.987607 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:51:59.990078 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:51:59.990063 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:52:00.036037 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:00.036012 2557 scope.go:117] "RemoveContainer" containerID="1026bb99d87a707517a7b2d7570aa3264ae6fcf069bb7fb5a4b4455799e9e95b" Jul 15 18:52:00.043692 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:00.043671 2557 scope.go:117] "RemoveContainer" containerID="21b223409d42573efeacb2134a590c819f2fbc543a9b52b4f88df8f3c1f050cf" Jul 15 18:52:17.022030 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.021977 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2"] Jul 15 18:52:17.022506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.022323 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06ea689-ffa4-44c4-93c4-c2db0bbc5724" containerName="revoke-keys" Jul 15 18:52:17.022506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.022334 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06ea689-ffa4-44c4-93c4-c2db0bbc5724" containerName="revoke-keys" Jul 15 18:52:17.022506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.022394 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06ea689-ffa4-44c4-93c4-c2db0bbc5724" containerName="revoke-keys" Jul 15 18:52:17.025328 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.025309 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:17.030724 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.030704 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:52:17.030995 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.030977 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:52:17.046076 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.046043 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2"] Jul 15 18:52:17.126854 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.126816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f533c1f-6d62-407b-bb7a-9afe233b1e9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2\" (UID: \"3f533c1f-6d62-407b-bb7a-9afe233b1e9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:17.227529 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.227489 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f533c1f-6d62-407b-bb7a-9afe233b1e9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2\" (UID: \"3f533c1f-6d62-407b-bb7a-9afe233b1e9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:17.228149 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.228127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f533c1f-6d62-407b-bb7a-9afe233b1e9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2\" (UID: \"3f533c1f-6d62-407b-bb7a-9afe233b1e9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:17.335625 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.335538 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:17.522724 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:17.522698 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2"] Jul 15 18:52:17.524329 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:52:17.524302 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f533c1f_6d62_407b_bb7a_9afe233b1e9a.slice/crio-3a040724f484d5a169c5e1e2d8b2b69192b5683859003fdce5188cde05a0f13e WatchSource:0}: Error finding container 3a040724f484d5a169c5e1e2d8b2b69192b5683859003fdce5188cde05a0f13e: Status 404 returned error can't find the container with id 3a040724f484d5a169c5e1e2d8b2b69192b5683859003fdce5188cde05a0f13e Jul 15 18:52:18.273166 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:18.273128 2557 generic.go:356] "Generic (PLEG): container finished" podID="3f533c1f-6d62-407b-bb7a-9afe233b1e9a" containerID="09cf052222c5ab6e04229c05917873f6efa9fd07f1f832ded90d561bc42d48bd" exitCode=0 Jul 15 18:52:18.273526 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:18.273210 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" event={"ID":"3f533c1f-6d62-407b-bb7a-9afe233b1e9a","Type":"ContainerDied","Data":"09cf052222c5ab6e04229c05917873f6efa9fd07f1f832ded90d561bc42d48bd"} Jul 15 18:52:18.273526 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:18.273266 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" event={"ID":"3f533c1f-6d62-407b-bb7a-9afe233b1e9a","Type":"ContainerStarted","Data":"3a040724f484d5a169c5e1e2d8b2b69192b5683859003fdce5188cde05a0f13e"} Jul 15 18:52:19.405073 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:19.405047 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:19.447891 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:19.447862 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f533c1f-6d62-407b-bb7a-9afe233b1e9a-maas-api-service-ca\") pod \"3f533c1f-6d62-407b-bb7a-9afe233b1e9a\" (UID: \"3f533c1f-6d62-407b-bb7a-9afe233b1e9a\") " Jul 15 18:52:19.448248 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:19.448218 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f533c1f-6d62-407b-bb7a-9afe233b1e9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f533c1f-6d62-407b-bb7a-9afe233b1e9a" (UID: "3f533c1f-6d62-407b-bb7a-9afe233b1e9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:52:19.548623 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:19.548532 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f533c1f-6d62-407b-bb7a-9afe233b1e9a-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:52:20.284192 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:20.284111 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" Jul 15 18:52:20.284192 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:20.284118 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2" event={"ID":"3f533c1f-6d62-407b-bb7a-9afe233b1e9a","Type":"ContainerDied","Data":"3a040724f484d5a169c5e1e2d8b2b69192b5683859003fdce5188cde05a0f13e"} Jul 15 18:52:20.284192 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:52:20.284159 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a040724f484d5a169c5e1e2d8b2b69192b5683859003fdce5188cde05a0f13e" Jul 15 18:53:00.067019 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:53:00.066946 2557 scope.go:117] "RemoveContainer" containerID="365e105d6bde2a725ae2f217b31645dad528d2942727b989c481e373d25b87c0" Jul 15 18:54:00.089296 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:54:00.089264 2557 scope.go:117] "RemoveContainer" containerID="e62ce3053eb93e16f820e0872e8d5684f32b9c11f968463306c0fc34d3576274" Jul 15 18:55:58.198940 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.198860 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t"] Jul 15 18:55:58.199369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.199235 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f533c1f-6d62-407b-bb7a-9afe233b1e9a" containerName="revoke-keys" Jul 15 18:55:58.199369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.199250 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f533c1f-6d62-407b-bb7a-9afe233b1e9a" containerName="revoke-keys" Jul 15 18:55:58.199369 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.199350 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f533c1f-6d62-407b-bb7a-9afe233b1e9a" containerName="revoke-keys" Jul 15 18:55:58.202360 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.202343 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:55:58.205015 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.204994 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:55:58.205129 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.205012 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:55:58.210087 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.210063 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t"] Jul 15 18:55:58.268968 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.268930 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t\" (UID: \"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:55:58.369916 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.369881 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t\" (UID: \"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:55:58.370503 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.370481 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t\" (UID: \"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:55:58.514410 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.514334 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:55:58.650914 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:58.650846 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t"] Jul 15 18:55:58.653135 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:55:58.653106 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f1c12b_b3ee_4c2b_8ba3_e5aafc80953d.slice/crio-ec76dc6fe47b25868538497af2a861fadc12bf79692b10ed6b4c94e77fb0344f WatchSource:0}: Error finding container ec76dc6fe47b25868538497af2a861fadc12bf79692b10ed6b4c94e77fb0344f: Status 404 returned error can't find the container with id ec76dc6fe47b25868538497af2a861fadc12bf79692b10ed6b4c94e77fb0344f Jul 15 18:55:59.296438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:59.296401 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" event={"ID":"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d","Type":"ContainerStarted","Data":"341dc00b4980d51d47d08e611867889ae8397662c32ebac6d34cd8c5e9037fa8"} Jul 15 18:55:59.296750 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:59.296465 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" event={"ID":"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d","Type":"ContainerStarted","Data":"ec76dc6fe47b25868538497af2a861fadc12bf79692b10ed6b4c94e77fb0344f"} Jul 15 18:55:59.315085 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:55:59.315041 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" podStartSLOduration=1.315024178 podStartE2EDuration="1.315024178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:55:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:55:59.314016198 +0000 UTC m=+2339.923829387" watchObservedRunningTime="2026-07-15 18:55:59.315024178 +0000 UTC m=+2339.924837365" Jul 15 18:56:00.302999 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:00.302964 2557 generic.go:356] "Generic (PLEG): container finished" podID="d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d" containerID="341dc00b4980d51d47d08e611867889ae8397662c32ebac6d34cd8c5e9037fa8" exitCode=0 Jul 15 18:56:00.303423 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:00.303047 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" event={"ID":"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d","Type":"ContainerDied","Data":"341dc00b4980d51d47d08e611867889ae8397662c32ebac6d34cd8c5e9037fa8"} Jul 15 18:56:01.434631 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:01.434607 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:56:01.600070 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:01.600031 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d-maas-api-service-ca\") pod \"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d\" (UID: \"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d\") " Jul 15 18:56:01.600434 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:01.600408 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d" (UID: "d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:56:01.701422 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:01.701384 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:56:02.313839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:02.313744 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" Jul 15 18:56:02.313839 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:02.313801 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t" event={"ID":"d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d","Type":"ContainerDied","Data":"ec76dc6fe47b25868538497af2a861fadc12bf79692b10ed6b4c94e77fb0344f"} Jul 15 18:56:02.314019 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:02.313847 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec76dc6fe47b25868538497af2a861fadc12bf79692b10ed6b4c94e77fb0344f" Jul 15 18:56:24.515935 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.515900 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v"] Jul 15 18:56:24.516418 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.516388 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d" containerName="revoke-keys" Jul 15 18:56:24.516418 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.516406 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d" containerName="revoke-keys" Jul 15 18:56:24.516527 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.516488 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d" containerName="revoke-keys" Jul 15 18:56:24.519717 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.519696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:24.524045 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.524023 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:56:24.524257 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.524239 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:56:24.536003 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.535977 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v"] Jul 15 18:56:24.584238 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.584206 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/192d3916-6f36-4788-b922-327555b03ab5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v\" (UID: \"192d3916-6f36-4788-b922-327555b03ab5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:24.685437 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.685397 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/192d3916-6f36-4788-b922-327555b03ab5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v\" (UID: \"192d3916-6f36-4788-b922-327555b03ab5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:24.686049 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.686028 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/192d3916-6f36-4788-b922-327555b03ab5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v\" (UID: \"192d3916-6f36-4788-b922-327555b03ab5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:24.829513 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.829430 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:24.971679 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:24.971652 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v"] Jul 15 18:56:24.973469 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:56:24.973439 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192d3916_6f36_4788_b922_327555b03ab5.slice/crio-bb514385d546938ffe930a7c83a61a84b9ca30e7a303c94b6e05d9ee7f7b7609 WatchSource:0}: Error finding container bb514385d546938ffe930a7c83a61a84b9ca30e7a303c94b6e05d9ee7f7b7609: Status 404 returned error can't find the container with id bb514385d546938ffe930a7c83a61a84b9ca30e7a303c94b6e05d9ee7f7b7609 Jul 15 18:56:25.420570 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:25.420534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" event={"ID":"192d3916-6f36-4788-b922-327555b03ab5","Type":"ContainerStarted","Data":"3012c3f5b4d86ef85c79894b18f900c1da79ce611577963f79329de521d3f216"} Jul 15 18:56:25.420751 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:25.420595 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" event={"ID":"192d3916-6f36-4788-b922-327555b03ab5","Type":"ContainerStarted","Data":"bb514385d546938ffe930a7c83a61a84b9ca30e7a303c94b6e05d9ee7f7b7609"} Jul 15 18:56:25.452536 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:25.452483 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" podStartSLOduration=1.452465034 podStartE2EDuration="1.452465034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:25.451677119 +0000 UTC m=+2366.061490309" watchObservedRunningTime="2026-07-15 18:56:25.452465034 +0000 UTC m=+2366.062278226" Jul 15 18:56:26.427079 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:26.427046 2557 generic.go:356] "Generic (PLEG): container finished" podID="192d3916-6f36-4788-b922-327555b03ab5" containerID="3012c3f5b4d86ef85c79894b18f900c1da79ce611577963f79329de521d3f216" exitCode=0 Jul 15 18:56:26.427455 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:26.427131 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" event={"ID":"192d3916-6f36-4788-b922-327555b03ab5","Type":"ContainerDied","Data":"3012c3f5b4d86ef85c79894b18f900c1da79ce611577963f79329de521d3f216"} Jul 15 18:56:27.568012 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:27.567989 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:27.611094 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:27.611061 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/192d3916-6f36-4788-b922-327555b03ab5-maas-api-service-ca\") pod \"192d3916-6f36-4788-b922-327555b03ab5\" (UID: \"192d3916-6f36-4788-b922-327555b03ab5\") " Jul 15 18:56:27.611463 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:27.611439 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192d3916-6f36-4788-b922-327555b03ab5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "192d3916-6f36-4788-b922-327555b03ab5" (UID: "192d3916-6f36-4788-b922-327555b03ab5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:56:27.712697 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:27.712610 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/192d3916-6f36-4788-b922-327555b03ab5-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:56:28.442973 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:28.442943 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" Jul 15 18:56:28.442973 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:28.442953 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v" event={"ID":"192d3916-6f36-4788-b922-327555b03ab5","Type":"ContainerDied","Data":"bb514385d546938ffe930a7c83a61a84b9ca30e7a303c94b6e05d9ee7f7b7609"} Jul 15 18:56:28.443184 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:28.443005 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb514385d546938ffe930a7c83a61a84b9ca30e7a303c94b6e05d9ee7f7b7609" Jul 15 18:56:51.310317 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.310279 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh"] Jul 15 18:56:51.310837 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.310628 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192d3916-6f36-4788-b922-327555b03ab5" containerName="revoke-keys" Jul 15 18:56:51.310837 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.310643 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="192d3916-6f36-4788-b922-327555b03ab5" containerName="revoke-keys" Jul 15 18:56:51.310837 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.310714 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="192d3916-6f36-4788-b922-327555b03ab5" containerName="revoke-keys" Jul 15 18:56:51.313707 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.313691 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.318480 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.318454 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a0cde7-openshift-default-dockercfg-95wl9\"" Jul 15 18:56:51.330780 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.330742 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh"] Jul 15 18:56:51.412966 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.412877 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-podinfo\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.412966 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.412935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-credential-socket\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413202 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413007 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-certs\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413202 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413075 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istiod-ca-cert\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413202 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413115 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-socket\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413202 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-token\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413445 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-envoy\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413445 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9p6\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-kube-api-access-nw9p6\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.413445 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.413341 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-data\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istiod-ca-cert\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514189 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-socket\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-token\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514240 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-envoy\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514264 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9p6\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-kube-api-access-nw9p6\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514302 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-data\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-podinfo\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-credential-socket\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514438 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-certs\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514736 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514684 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-socket\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514736 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514724 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-certs\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514844 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514800 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-data\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.514879 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514857 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-credential-socket\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.515018 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.514998 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istiod-ca-cert\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.516664 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.516645 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-envoy\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.516937 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.516917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-podinfo\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.523113 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.523091 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-token\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.523225 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.523167 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9p6\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-kube-api-access-nw9p6\") pod \"e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.623763 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.623731 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:51.785778 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.785740 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh"] Jul 15 18:56:51.787237 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:56:51.787206 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297f3ede_cd87_4435_9b07_8f1a31f8a8be.slice/crio-fd62ce2a6ab2004d471fef31d75a7fb5963db2b9fa7e4e0faa0c88e7d1652cfc WatchSource:0}: Error finding container fd62ce2a6ab2004d471fef31d75a7fb5963db2b9fa7e4e0faa0c88e7d1652cfc: Status 404 returned error can't find the container with id fd62ce2a6ab2004d471fef31d75a7fb5963db2b9fa7e4e0faa0c88e7d1652cfc Jul 15 18:56:51.789280 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.789247 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:56:51.789382 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.789314 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:56:51.789382 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:51.789343 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:56:52.556348 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:52.556312 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" event={"ID":"297f3ede-cd87-4435-9b07-8f1a31f8a8be","Type":"ContainerStarted","Data":"9ce2b6c274555514174fecd9941450a478b8de58132ed96246d9bf329756d7ac"} Jul 15 18:56:52.556712 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:52.556372 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" event={"ID":"297f3ede-cd87-4435-9b07-8f1a31f8a8be","Type":"ContainerStarted","Data":"fd62ce2a6ab2004d471fef31d75a7fb5963db2b9fa7e4e0faa0c88e7d1652cfc"} Jul 15 18:56:52.582440 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:52.582394 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" podStartSLOduration=1.582377422 podStartE2EDuration="1.582377422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:56:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:56:52.579812577 +0000 UTC m=+2393.189625768" watchObservedRunningTime="2026-07-15 18:56:52.582377422 +0000 UTC m=+2393.192190610" Jul 15 18:56:52.624391 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:52.624359 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:52.629316 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:52.629291 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:53.561818 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:53.561787 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:56:53.562703 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:56:53.562684 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:00.011633 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:00.011511 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:57:00.022930 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:00.014423 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 18:57:00.022930 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:00.017745 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:57:00.022930 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:00.020127 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 18:57:07.645619 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:07.645580 2557 generic.go:356] "Generic (PLEG): container finished" podID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerID="9ce2b6c274555514174fecd9941450a478b8de58132ed96246d9bf329756d7ac" exitCode=0 Jul 15 18:57:07.646062 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:07.645655 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" event={"ID":"297f3ede-cd87-4435-9b07-8f1a31f8a8be","Type":"ContainerDied","Data":"9ce2b6c274555514174fecd9941450a478b8de58132ed96246d9bf329756d7ac"} Jul 15 18:57:07.646106 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:07.646074 2557 scope.go:117] "RemoveContainer" containerID="9ce2b6c274555514174fecd9941450a478b8de58132ed96246d9bf329756d7ac" Jul 15 18:57:07.647024 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:07.646998 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:57:07.647087 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:07.647063 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:57:07.647125 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:07.647095 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:57:08.562674 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:08.562635 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:08.652751 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:08.652708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" event={"ID":"297f3ede-cd87-4435-9b07-8f1a31f8a8be","Type":"ContainerStarted","Data":"8e8923fc91097f66cd88ab971893fb55ba77f0b1bf42f83b151ee0101a396592"} Jul 15 18:57:09.624183 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:09.624142 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:09.625405 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:09.625378 2557 patch_prober.go:28] interesting pod/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 15 18:57:09.625541 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:09.625438 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 15 18:57:10.624402 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:10.624363 2557 patch_prober.go:28] interesting pod/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 15 18:57:10.624840 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:10.624420 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 15 18:57:11.624460 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:11.624419 2557 patch_prober.go:28] interesting pod/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 15 18:57:11.624899 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:11.624476 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 15 18:57:12.628975 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:12.628942 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:12.629482 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:12.629344 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:12.629942 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:12.629922 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:15.873787 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:15.873745 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6"] Jul 15 18:57:15.876171 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:15.876155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:15.879252 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:15.879229 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:57:15.879375 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:15.879355 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:57:15.887392 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:15.887368 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6"] Jul 15 18:57:15.954397 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:15.954355 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fef1-15cd-4f03-a9ea-21562a1c3e62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6\" (UID: \"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:16.055885 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.055842 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fef1-15cd-4f03-a9ea-21562a1c3e62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6\" (UID: \"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:16.056496 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.056476 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fef1-15cd-4f03-a9ea-21562a1c3e62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6\" (UID: \"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:16.187624 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.187536 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:16.322446 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.322330 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6"] Jul 15 18:57:16.325321 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:57:16.325297 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca7fef1_15cd_4f03_a9ea_21562a1c3e62.slice/crio-ff115f5aa851f73cb6cc2c3d8f6421550f93081be21c76ec0f96f57e26536174 WatchSource:0}: Error finding container ff115f5aa851f73cb6cc2c3d8f6421550f93081be21c76ec0f96f57e26536174: Status 404 returned error can't find the container with id ff115f5aa851f73cb6cc2c3d8f6421550f93081be21c76ec0f96f57e26536174 Jul 15 18:57:16.691187 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.691152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" event={"ID":"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62","Type":"ContainerStarted","Data":"873bbd4555cc0985d4e15c8268d1fe1bdb9e483f54719f6962870b2f3735ee08"} Jul 15 18:57:16.691354 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.691204 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" event={"ID":"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62","Type":"ContainerStarted","Data":"ff115f5aa851f73cb6cc2c3d8f6421550f93081be21c76ec0f96f57e26536174"} Jul 15 18:57:16.711081 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:16.711018 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" podStartSLOduration=1.710995535 podStartE2EDuration="1.710995535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:57:16.708361543 +0000 UTC m=+2417.318174747" watchObservedRunningTime="2026-07-15 18:57:16.710995535 +0000 UTC m=+2417.320808724" Jul 15 18:57:17.697578 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:17.697540 2557 generic.go:356] "Generic (PLEG): container finished" podID="7ca7fef1-15cd-4f03-a9ea-21562a1c3e62" containerID="873bbd4555cc0985d4e15c8268d1fe1bdb9e483f54719f6962870b2f3735ee08" exitCode=0 Jul 15 18:57:17.698061 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:17.697630 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" event={"ID":"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62","Type":"ContainerDied","Data":"873bbd4555cc0985d4e15c8268d1fe1bdb9e483f54719f6962870b2f3735ee08"} Jul 15 18:57:18.832130 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:18.832101 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:18.883527 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:18.883475 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fef1-15cd-4f03-a9ea-21562a1c3e62-maas-api-service-ca\") pod \"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62\" (UID: \"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62\") " Jul 15 18:57:18.883876 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:18.883850 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ca7fef1-15cd-4f03-a9ea-21562a1c3e62-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ca7fef1-15cd-4f03-a9ea-21562a1c3e62" (UID: "7ca7fef1-15cd-4f03-a9ea-21562a1c3e62"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:57:18.985128 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:18.985041 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ca7fef1-15cd-4f03-a9ea-21562a1c3e62-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:19.708858 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:19.708822 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" event={"ID":"7ca7fef1-15cd-4f03-a9ea-21562a1c3e62","Type":"ContainerDied","Data":"ff115f5aa851f73cb6cc2c3d8f6421550f93081be21c76ec0f96f57e26536174"} Jul 15 18:57:19.709027 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:19.708874 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff115f5aa851f73cb6cc2c3d8f6421550f93081be21c76ec0f96f57e26536174" Jul 15 18:57:19.709027 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:19.708895 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6" Jul 15 18:57:41.721046 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:41.720956 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh"] Jul 15 18:57:41.721590 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:41.721220 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" containerID="cri-o://8e8923fc91097f66cd88ab971893fb55ba77f0b1bf42f83b151ee0101a396592" gracePeriod=30 Jul 15 18:57:42.139951 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.139913 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2"] Jul 15 18:57:42.140335 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.140317 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca7fef1-15cd-4f03-a9ea-21562a1c3e62" containerName="revoke-keys" Jul 15 18:57:42.140423 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.140337 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca7fef1-15cd-4f03-a9ea-21562a1c3e62" containerName="revoke-keys" Jul 15 18:57:42.140474 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.140446 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca7fef1-15cd-4f03-a9ea-21562a1c3e62" containerName="revoke-keys" Jul 15 18:57:42.142352 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.142331 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:42.146638 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.146621 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 18:57:42.146746 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.146704 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 18:57:42.154620 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.154599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2"] Jul 15 18:57:42.283433 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.283385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d86cff8b-9b6a-484d-879b-49e5d5c614a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2\" (UID: \"d86cff8b-9b6a-484d-879b-49e5d5c614a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:42.384830 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.384794 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d86cff8b-9b6a-484d-879b-49e5d5c614a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2\" (UID: \"d86cff8b-9b6a-484d-879b-49e5d5c614a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:42.385403 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.385385 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d86cff8b-9b6a-484d-879b-49e5d5c614a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2\" (UID: \"d86cff8b-9b6a-484d-879b-49e5d5c614a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:42.453372 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.453286 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:42.589974 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.589949 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2"] Jul 15 18:57:42.591471 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:57:42.591441 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86cff8b_9b6a_484d_879b_49e5d5c614a2.slice/crio-7079bab1cb1ccd2d2c6a77f48681d9efe855afb124ad57380723a553a0791280 WatchSource:0}: Error finding container 7079bab1cb1ccd2d2c6a77f48681d9efe855afb124ad57380723a553a0791280: Status 404 returned error can't find the container with id 7079bab1cb1ccd2d2c6a77f48681d9efe855afb124ad57380723a553a0791280 Jul 15 18:57:42.631335 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.631299 2557 patch_prober.go:28] interesting pod/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 15 18:57:42.631455 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.631360 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 18:57:42.817852 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:42.817824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" event={"ID":"d86cff8b-9b6a-484d-879b-49e5d5c614a2","Type":"ContainerStarted","Data":"7079bab1cb1ccd2d2c6a77f48681d9efe855afb124ad57380723a553a0791280"} Jul 15 18:57:43.824608 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:43.824575 2557 generic.go:356] "Generic (PLEG): container finished" podID="d86cff8b-9b6a-484d-879b-49e5d5c614a2" containerID="cc3978a20ee9ce9782e9375d1b940b779facd5b59c038cb911423ee8e83a4a82" exitCode=0 Jul 15 18:57:43.825004 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:43.824662 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" event={"ID":"d86cff8b-9b6a-484d-879b-49e5d5c614a2","Type":"ContainerDied","Data":"cc3978a20ee9ce9782e9375d1b940b779facd5b59c038cb911423ee8e83a4a82"} Jul 15 18:57:44.958301 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:44.958277 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:45.110862 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:45.110830 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d86cff8b-9b6a-484d-879b-49e5d5c614a2-maas-api-service-ca\") pod \"d86cff8b-9b6a-484d-879b-49e5d5c614a2\" (UID: \"d86cff8b-9b6a-484d-879b-49e5d5c614a2\") " Jul 15 18:57:45.111192 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:45.111168 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d86cff8b-9b6a-484d-879b-49e5d5c614a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d86cff8b-9b6a-484d-879b-49e5d5c614a2" (UID: "d86cff8b-9b6a-484d-879b-49e5d5c614a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:57:45.212658 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:45.212617 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d86cff8b-9b6a-484d-879b-49e5d5c614a2-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:45.835353 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:45.835324 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" Jul 15 18:57:45.835526 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:45.835374 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2" event={"ID":"d86cff8b-9b6a-484d-879b-49e5d5c614a2","Type":"ContainerDied","Data":"7079bab1cb1ccd2d2c6a77f48681d9efe855afb124ad57380723a553a0791280"} Jul 15 18:57:45.835526 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:45.835444 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7079bab1cb1ccd2d2c6a77f48681d9efe855afb124ad57380723a553a0791280" Jul 15 18:57:46.846291 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:46.846263 2557 generic.go:356] "Generic (PLEG): container finished" podID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerID="8e8923fc91097f66cd88ab971893fb55ba77f0b1bf42f83b151ee0101a396592" exitCode=0 Jul 15 18:57:46.846699 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:46.846329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" event={"ID":"297f3ede-cd87-4435-9b07-8f1a31f8a8be","Type":"ContainerDied","Data":"8e8923fc91097f66cd88ab971893fb55ba77f0b1bf42f83b151ee0101a396592"} Jul 15 18:57:46.846699 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:46.846386 2557 scope.go:117] "RemoveContainer" containerID="9ce2b6c274555514174fecd9941450a478b8de58132ed96246d9bf329756d7ac" Jul 15 18:57:46.969015 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:46.968990 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:47.129093 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129058 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-credential-socket\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129281 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129109 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-socket\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129281 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129169 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-certs\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129281 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129185 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-data\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129281 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129218 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-podinfo\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129281 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129243 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-token\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129281 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129269 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw9p6\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-kube-api-access-nw9p6\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129304 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-envoy\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129356 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istiod-ca-cert\") pod \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\" (UID: \"297f3ede-cd87-4435-9b07-8f1a31f8a8be\") " Jul 15 18:57:47.129594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129397 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:47.129594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129522 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:47.129594 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129536 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:47.129850 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129693 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-socket\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.129850 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129714 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-workload-certs\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.129850 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129728 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-credential-socket\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.129992 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.129861 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 18:57:47.130202 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.130171 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-data" (OuterVolumeSpecName: "istio-data") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:47.131699 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.131674 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 18:57:47.131699 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.131685 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-token" (OuterVolumeSpecName: "istio-token") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:57:47.131851 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.131699 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-kube-api-access-nw9p6" (OuterVolumeSpecName: "kube-api-access-nw9p6") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "kube-api-access-nw9p6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 18:57:47.131851 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.131699 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "297f3ede-cd87-4435-9b07-8f1a31f8a8be" (UID: "297f3ede-cd87-4435-9b07-8f1a31f8a8be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 18:57:47.230571 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.230533 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istiod-ca-cert\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.230571 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.230567 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-data\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.230571 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.230580 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-podinfo\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.230854 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.230591 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-token\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.230854 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.230604 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9p6\" (UniqueName: \"kubernetes.io/projected/297f3ede-cd87-4435-9b07-8f1a31f8a8be-kube-api-access-nw9p6\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.230854 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.230616 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/297f3ede-cd87-4435-9b07-8f1a31f8a8be-istio-envoy\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 18:57:47.853745 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.853712 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" Jul 15 18:57:47.854205 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.853747 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh" event={"ID":"297f3ede-cd87-4435-9b07-8f1a31f8a8be","Type":"ContainerDied","Data":"fd62ce2a6ab2004d471fef31d75a7fb5963db2b9fa7e4e0faa0c88e7d1652cfc"} Jul 15 18:57:47.854205 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.853815 2557 scope.go:117] "RemoveContainer" containerID="8e8923fc91097f66cd88ab971893fb55ba77f0b1bf42f83b151ee0101a396592" Jul 15 18:57:47.877287 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.877251 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh"] Jul 15 18:57:47.880922 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.880893 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a0cde7-openshift-default-689dc94df5-jxgkh"] Jul 15 18:57:47.998622 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:57:47.998585 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" path="/var/lib/kubelet/pods/297f3ede-cd87-4435-9b07-8f1a31f8a8be/volumes" Jul 15 18:58:09.080571 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080535 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7"] Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080882 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080894 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080902 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d86cff8b-9b6a-484d-879b-49e5d5c614a2" containerName="revoke-keys" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080907 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d86cff8b-9b6a-484d-879b-49e5d5c614a2" containerName="revoke-keys" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080918 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080924 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080987 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.080994 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="297f3ede-cd87-4435-9b07-8f1a31f8a8be" containerName="istio-proxy" Jul 15 18:58:09.081040 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.081000 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d86cff8b-9b6a-484d-879b-49e5d5c614a2" containerName="revoke-keys" Jul 15 18:58:09.084020 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.084002 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.088269 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.088253 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-401e9f-openshift-default-dockercfg-btqvk\"" Jul 15 18:58:09.103323 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.103298 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7"] Jul 15 18:58:09.105050 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105019 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbftj\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-kube-api-access-vbftj\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105180 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105065 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-socket\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105180 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-credential-socket\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105180 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-data\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105180 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105162 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-envoy\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105389 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105215 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-certs\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105389 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105262 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-podinfo\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105389 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105320 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-token\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.105389 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.105358 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istiod-ca-cert\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206310 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206272 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-certs\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-podinfo\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206359 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-token\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206506 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istiod-ca-cert\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206696 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbftj\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-kube-api-access-vbftj\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206696 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206554 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-socket\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206696 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-credential-socket\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206696 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206626 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-data\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206696 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206655 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-envoy\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206929 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206719 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-certs\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.206929 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206913 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-socket\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.207010 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.206975 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-credential-socket\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.207079 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.207063 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-data\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.207234 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.207207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istiod-ca-cert\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.208991 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.208965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-envoy\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.209204 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.209189 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-podinfo\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.217201 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.217168 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-token\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.218457 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.218435 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbftj\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-kube-api-access-vbftj\") pod \"e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.393619 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.393576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:09.547196 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.547166 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7"] Jul 15 18:58:09.549736 ip-10-0-130-233 kubenswrapper[2557]: W0715 18:58:09.549707 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8921bc_17b2_4392_b811_f745a0ef7ebd.slice/crio-d261d32251d9d6352a4331ebae8194384f4143910903f5740a5309aa5336cbb6 WatchSource:0}: Error finding container d261d32251d9d6352a4331ebae8194384f4143910903f5740a5309aa5336cbb6: Status 404 returned error can't find the container with id d261d32251d9d6352a4331ebae8194384f4143910903f5740a5309aa5336cbb6 Jul 15 18:58:09.551985 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.551950 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:58:09.552063 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.552024 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:58:09.552063 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.552052 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:58:09.961239 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.961205 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" event={"ID":"9f8921bc-17b2-4392-b811-f745a0ef7ebd","Type":"ContainerStarted","Data":"230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3"} Jul 15 18:58:09.961429 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.961260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" event={"ID":"9f8921bc-17b2-4392-b811-f745a0ef7ebd","Type":"ContainerStarted","Data":"d261d32251d9d6352a4331ebae8194384f4143910903f5740a5309aa5336cbb6"} Jul 15 18:58:09.989100 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:09.989043 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podStartSLOduration=0.989024145 podStartE2EDuration="989.024145ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 18:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 18:58:09.985953182 +0000 UTC m=+2470.595766372" watchObservedRunningTime="2026-07-15 18:58:09.989024145 +0000 UTC m=+2470.598837334" Jul 15 18:58:10.393926 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:10.393884 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:10.398842 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:10.398818 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:10.966145 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:10.966109 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:10.967030 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:10.967011 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:23.470054 ip-10-0-130-233 kubenswrapper[2557]: E0715 18:58:23.470014 2557 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8921bc_17b2_4392_b811_f745a0ef7ebd.slice/crio-conmon-230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3.scope\": RecentStats: unable to find data in memory cache]" Jul 15 18:58:24.051603 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:24.051570 2557 generic.go:356] "Generic (PLEG): container finished" podID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerID="230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3" exitCode=0 Jul 15 18:58:24.051800 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:24.051640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" event={"ID":"9f8921bc-17b2-4392-b811-f745a0ef7ebd","Type":"ContainerDied","Data":"230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3"} Jul 15 18:58:24.052029 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:24.052014 2557 scope.go:117] "RemoveContainer" containerID="230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3" Jul 15 18:58:24.052997 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:24.052972 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:58:24.053057 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:24.053036 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:58:24.053091 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:24.053067 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 15 18:58:25.058878 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:25.058836 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" event={"ID":"9f8921bc-17b2-4392-b811-f745a0ef7ebd","Type":"ContainerStarted","Data":"9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64"} Jul 15 18:58:25.395319 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:25.395233 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:25.395675 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:25.395650 2557 patch_prober.go:28] interesting pod/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 18:58:25.395878 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:25.395853 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 18:58:25.966870 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:25.966828 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:26.394483 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:26.394442 2557 patch_prober.go:28] interesting pod/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 18:58:26.395129 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:26.395092 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 18:58:27.394664 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:27.394620 2557 patch_prober.go:28] interesting pod/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 18:58:27.395280 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:27.395248 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 18:58:28.394778 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:28.394718 2557 patch_prober.go:28] interesting pod/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 18:58:28.395286 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:28.394824 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 18:58:29.398243 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:29.398208 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 18:58:29.399105 ip-10-0-130-233 kubenswrapper[2557]: I0715 18:58:29.399086 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 19:00:00.140576 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.140538 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl"] Jul 15 19:00:00.143816 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.143790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2"] Jul 15 19:00:00.143931 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.143899 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" Jul 15 19:00:00.147035 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.147012 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 19:00:00.147231 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.147212 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" Jul 15 19:00:00.158412 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.158386 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" Jul 15 19:00:00.159176 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.159002 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl"] Jul 15 19:00:00.162261 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.162236 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2"] Jul 15 19:00:00.167567 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.167542 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" Jul 15 19:00:00.316652 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.316582 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl"] Jul 15 19:00:00.319169 ip-10-0-130-233 kubenswrapper[2557]: W0715 19:00:00.319126 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47029f8_bfa8_4eb0_b54b_4c29baebbedc.slice/crio-e2d7736289158d70f87e21735e38179e2c639f5a56fce58ab7b5f8fa9640f890 WatchSource:0}: Error finding container e2d7736289158d70f87e21735e38179e2c639f5a56fce58ab7b5f8fa9640f890: Status 404 returned error can't find the container with id e2d7736289158d70f87e21735e38179e2c639f5a56fce58ab7b5f8fa9640f890 Jul 15 19:00:00.339391 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.339337 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2"] Jul 15 19:00:00.343411 ip-10-0-130-233 kubenswrapper[2557]: W0715 19:00:00.343379 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4effdf43_d7a5_4dd1_bd3f_4be06aadf131.slice/crio-f51ff3b9493b0add54350c97dc64715166855d3973674b936f5c8bfca47bf5ba WatchSource:0}: Error finding container f51ff3b9493b0add54350c97dc64715166855d3973674b936f5c8bfca47bf5ba: Status 404 returned error can't find the container with id f51ff3b9493b0add54350c97dc64715166855d3973674b936f5c8bfca47bf5ba Jul 15 19:00:00.537053 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.537007 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" event={"ID":"4effdf43-d7a5-4dd1-bd3f-4be06aadf131","Type":"ContainerStarted","Data":"4562b07303cf73cc55f8dd221ffb3bafa27b90796495b33d608af5e33dfb5dec"} Jul 15 19:00:00.537249 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.537073 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" event={"ID":"4effdf43-d7a5-4dd1-bd3f-4be06aadf131","Type":"ContainerStarted","Data":"f51ff3b9493b0add54350c97dc64715166855d3973674b936f5c8bfca47bf5ba"} Jul 15 19:00:00.538116 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.538095 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" event={"ID":"d47029f8-bfa8-4eb0-b54b-4c29baebbedc","Type":"ContainerStarted","Data":"e2d7736289158d70f87e21735e38179e2c639f5a56fce58ab7b5f8fa9640f890"} Jul 15 19:00:00.554030 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:00.553980 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" podStartSLOduration=0.553962552 podStartE2EDuration="553.962552ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:00:00.551611537 +0000 UTC m=+2581.161424728" watchObservedRunningTime="2026-07-15 19:00:00.553962552 +0000 UTC m=+2581.163775740" Jul 15 19:00:01.544106 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:01.544011 2557 generic.go:356] "Generic (PLEG): container finished" podID="d47029f8-bfa8-4eb0-b54b-4c29baebbedc" containerID="ecf4f8697ebc79cc16b26c1559b6013103d3b7373df851467bd4ab09080a6445" exitCode=0 Jul 15 19:00:01.544552 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:01.544097 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" event={"ID":"d47029f8-bfa8-4eb0-b54b-4c29baebbedc","Type":"ContainerDied","Data":"ecf4f8697ebc79cc16b26c1559b6013103d3b7373df851467bd4ab09080a6445"} Jul 15 19:00:01.545688 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:01.545665 2557 generic.go:356] "Generic (PLEG): container finished" podID="4effdf43-d7a5-4dd1-bd3f-4be06aadf131" containerID="4562b07303cf73cc55f8dd221ffb3bafa27b90796495b33d608af5e33dfb5dec" exitCode=0 Jul 15 19:00:01.545815 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:01.545733 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" event={"ID":"4effdf43-d7a5-4dd1-bd3f-4be06aadf131","Type":"ContainerDied","Data":"4562b07303cf73cc55f8dd221ffb3bafa27b90796495b33d608af5e33dfb5dec"} Jul 15 19:00:02.703818 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:02.703762 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" Jul 15 19:00:02.707087 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:02.707062 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" Jul 15 19:00:03.557138 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:03.557101 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" Jul 15 19:00:03.557348 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:03.557135 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl" event={"ID":"d47029f8-bfa8-4eb0-b54b-4c29baebbedc","Type":"ContainerDied","Data":"e2d7736289158d70f87e21735e38179e2c639f5a56fce58ab7b5f8fa9640f890"} Jul 15 19:00:03.557348 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:03.557187 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2d7736289158d70f87e21735e38179e2c639f5a56fce58ab7b5f8fa9640f890" Jul 15 19:00:03.558506 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:03.558485 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" Jul 15 19:00:03.558506 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:03.558492 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2" event={"ID":"4effdf43-d7a5-4dd1-bd3f-4be06aadf131","Type":"ContainerDied","Data":"f51ff3b9493b0add54350c97dc64715166855d3973674b936f5c8bfca47bf5ba"} Jul 15 19:00:03.558694 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:00:03.558532 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f51ff3b9493b0add54350c97dc64715166855d3973674b936f5c8bfca47bf5ba" Jul 15 19:01:14.501099 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501061 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s"] Jul 15 19:01:14.501593 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501577 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47029f8-bfa8-4eb0-b54b-4c29baebbedc" containerName="cleanup" Jul 15 19:01:14.501640 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501596 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47029f8-bfa8-4eb0-b54b-4c29baebbedc" containerName="cleanup" Jul 15 19:01:14.501640 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501614 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4effdf43-d7a5-4dd1-bd3f-4be06aadf131" containerName="cleanup" Jul 15 19:01:14.501640 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501623 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4effdf43-d7a5-4dd1-bd3f-4be06aadf131" containerName="cleanup" Jul 15 19:01:14.501728 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501712 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4effdf43-d7a5-4dd1-bd3f-4be06aadf131" containerName="cleanup" Jul 15 19:01:14.501788 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.501726 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47029f8-bfa8-4eb0-b54b-4c29baebbedc" containerName="cleanup" Jul 15 19:01:14.505873 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.505848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.509702 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.509678 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-401e9f\"/\"kube-root-ca.crt\"" Jul 15 19:01:14.509988 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.509690 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-401e9f\"/\"default-dockercfg-7zfmv\"" Jul 15 19:01:14.510107 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.509690 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-401e9f\"/\"sub-test-model-401e9f-kserve-self-signed-certs\"" Jul 15 19:01:14.510107 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.509734 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-401e9f\"/\"openshift-service-ca.crt\"" Jul 15 19:01:14.515825 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.515794 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s"] Jul 15 19:01:14.642820 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.642754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-tmp-dir\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.642995 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.642832 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzt8q\" (UniqueName: \"kubernetes.io/projected/176f4a7a-013d-4785-999e-63ddc18e192f-kube-api-access-hzt8q\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.642995 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.642861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-dshm\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.642995 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.642890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-home\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.642995 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.642924 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-model-cache\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.642995 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.642940 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/176f4a7a-013d-4785-999e-63ddc18e192f-tls-certs\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744401 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744356 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-tmp-dir\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744401 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzt8q\" (UniqueName: \"kubernetes.io/projected/176f4a7a-013d-4785-999e-63ddc18e192f-kube-api-access-hzt8q\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744625 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-dshm\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744625 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-home\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744625 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-model-cache\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744819 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/176f4a7a-013d-4785-999e-63ddc18e192f-tls-certs\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744888 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744867 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-tmp-dir\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.744978 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.744961 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-home\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.745079 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.745059 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-model-cache\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.746738 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.746719 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-dshm\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.747128 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.747106 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/176f4a7a-013d-4785-999e-63ddc18e192f-tls-certs\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.754305 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.754236 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzt8q\" (UniqueName: \"kubernetes.io/projected/176f4a7a-013d-4785-999e-63ddc18e192f-kube-api-access-hzt8q\") pod \"sub-test-model-401e9f-kserve-59855b8957-xfq7s\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.817952 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.817912 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:14.957422 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:14.957231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s"] Jul 15 19:01:14.960181 ip-10-0-130-233 kubenswrapper[2557]: W0715 19:01:14.960153 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod176f4a7a_013d_4785_999e_63ddc18e192f.slice/crio-2ec67bcd99908053670a9338ee6552eeb0456f40c333d3b3a4cef56488ef329c WatchSource:0}: Error finding container 2ec67bcd99908053670a9338ee6552eeb0456f40c333d3b3a4cef56488ef329c: Status 404 returned error can't find the container with id 2ec67bcd99908053670a9338ee6552eeb0456f40c333d3b3a4cef56488ef329c Jul 15 19:01:15.918424 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:15.918385 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" event={"ID":"176f4a7a-013d-4785-999e-63ddc18e192f","Type":"ContainerStarted","Data":"addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07"} Jul 15 19:01:15.918928 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:15.918431 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" event={"ID":"176f4a7a-013d-4785-999e-63ddc18e192f","Type":"ContainerStarted","Data":"2ec67bcd99908053670a9338ee6552eeb0456f40c333d3b3a4cef56488ef329c"} Jul 15 19:01:15.937198 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:15.937140 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" podStartSLOduration=1.9371228409999999 podStartE2EDuration="1.937122841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:01:15.935883963 +0000 UTC m=+2656.545697175" watchObservedRunningTime="2026-07-15 19:01:15.937122841 +0000 UTC m=+2656.546936028" Jul 15 19:01:24.818512 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:24.818469 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:24.819057 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:24.818825 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:24.831362 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:24.831336 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:01:24.975139 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:01:24.975103 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:02:00.046921 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:00.046845 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 19:02:00.050611 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:00.050587 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 19:02:00.052787 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:00.052754 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 19:02:00.056502 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:00.056482 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlr77_18c21857-b3f7-4582-afea-13f190c60530/ovn-acl-logging/0.log" Jul 15 19:02:34.968657 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:34.968621 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s"] Jul 15 19:02:49.989220 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:49.989177 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" podUID="176f4a7a-013d-4785-999e-63ddc18e192f" containerName="main" containerID="cri-o://addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07" gracePeriod=45 Jul 15 19:02:50.245886 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.245812 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:02:50.321875 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.321836 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-model-cache\") pod \"176f4a7a-013d-4785-999e-63ddc18e192f\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " Jul 15 19:02:50.321875 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.321888 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-home\") pod \"176f4a7a-013d-4785-999e-63ddc18e192f\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " Jul 15 19:02:50.322123 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.321917 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzt8q\" (UniqueName: \"kubernetes.io/projected/176f4a7a-013d-4785-999e-63ddc18e192f-kube-api-access-hzt8q\") pod \"176f4a7a-013d-4785-999e-63ddc18e192f\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " Jul 15 19:02:50.322123 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.321957 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-tmp-dir\") pod \"176f4a7a-013d-4785-999e-63ddc18e192f\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " Jul 15 19:02:50.322123 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322110 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-dshm\") pod \"176f4a7a-013d-4785-999e-63ddc18e192f\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " Jul 15 19:02:50.322450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322163 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/176f4a7a-013d-4785-999e-63ddc18e192f-tls-certs\") pod \"176f4a7a-013d-4785-999e-63ddc18e192f\" (UID: \"176f4a7a-013d-4785-999e-63ddc18e192f\") " Jul 15 19:02:50.322450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322155 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-home" (OuterVolumeSpecName: "home") pod "176f4a7a-013d-4785-999e-63ddc18e192f" (UID: "176f4a7a-013d-4785-999e-63ddc18e192f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:02:50.322450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322173 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-model-cache" (OuterVolumeSpecName: "model-cache") pod "176f4a7a-013d-4785-999e-63ddc18e192f" (UID: "176f4a7a-013d-4785-999e-63ddc18e192f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:02:50.322450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322184 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "176f4a7a-013d-4785-999e-63ddc18e192f" (UID: "176f4a7a-013d-4785-999e-63ddc18e192f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:02:50.322450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322424 2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-model-cache\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:02:50.322450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322444 2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-home\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:02:50.322654 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.322457 2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-tmp-dir\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:02:50.324269 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.324245 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-dshm" (OuterVolumeSpecName: "dshm") pod "176f4a7a-013d-4785-999e-63ddc18e192f" (UID: "176f4a7a-013d-4785-999e-63ddc18e192f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:02:50.324352 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.324289 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/176f4a7a-013d-4785-999e-63ddc18e192f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "176f4a7a-013d-4785-999e-63ddc18e192f" (UID: "176f4a7a-013d-4785-999e-63ddc18e192f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:02:50.324406 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.324373 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/176f4a7a-013d-4785-999e-63ddc18e192f-kube-api-access-hzt8q" (OuterVolumeSpecName: "kube-api-access-hzt8q") pod "176f4a7a-013d-4785-999e-63ddc18e192f" (UID: "176f4a7a-013d-4785-999e-63ddc18e192f"). InnerVolumeSpecName "kube-api-access-hzt8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:02:50.395972 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.395938 2557 generic.go:356] "Generic (PLEG): container finished" podID="176f4a7a-013d-4785-999e-63ddc18e192f" containerID="addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07" exitCode=0 Jul 15 19:02:50.395972 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.395976 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" Jul 15 19:02:50.396195 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.396018 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" event={"ID":"176f4a7a-013d-4785-999e-63ddc18e192f","Type":"ContainerDied","Data":"addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07"} Jul 15 19:02:50.396195 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.396057 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s" event={"ID":"176f4a7a-013d-4785-999e-63ddc18e192f","Type":"ContainerDied","Data":"2ec67bcd99908053670a9338ee6552eeb0456f40c333d3b3a4cef56488ef329c"} Jul 15 19:02:50.396195 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.396075 2557 scope.go:117] "RemoveContainer" containerID="addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07" Jul 15 19:02:50.407463 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.407436 2557 scope.go:117] "RemoveContainer" containerID="addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07" Jul 15 19:02:50.407895 ip-10-0-130-233 kubenswrapper[2557]: E0715 19:02:50.407873 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07\": container with ID starting with addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07 not found: ID does not exist" containerID="addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07" Jul 15 19:02:50.407999 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.407903 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07"} err="failed to get container status \"addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07\": rpc error: code = NotFound desc = could not find container \"addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07\": container with ID starting with addb60aaff63eceb20dbd2ec688527f133fcf73dbe6faedf316bb541e60f2e07 not found: ID does not exist" Jul 15 19:02:50.417781 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.417738 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s"] Jul 15 19:02:50.421360 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.421334 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-401e9f/sub-test-model-401e9f-kserve-59855b8957-xfq7s"] Jul 15 19:02:50.423265 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.423247 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzt8q\" (UniqueName: \"kubernetes.io/projected/176f4a7a-013d-4785-999e-63ddc18e192f-kube-api-access-hzt8q\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:02:50.423316 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.423269 2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/176f4a7a-013d-4785-999e-63ddc18e192f-dshm\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:02:50.423316 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:50.423280 2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/176f4a7a-013d-4785-999e-63ddc18e192f-tls-certs\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:02:51.997918 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:02:51.997875 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="176f4a7a-013d-4785-999e-63ddc18e192f" path="/var/lib/kubelet/pods/176f4a7a-013d-4785-999e-63ddc18e192f/volumes" Jul 15 19:04:44.023260 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.023228 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds"] Jul 15 19:04:44.023850 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.023556 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="176f4a7a-013d-4785-999e-63ddc18e192f" containerName="main" Jul 15 19:04:44.023850 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.023567 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="176f4a7a-013d-4785-999e-63ddc18e192f" containerName="main" Jul 15 19:04:44.023850 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.023653 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="176f4a7a-013d-4785-999e-63ddc18e192f" containerName="main" Jul 15 19:04:44.026464 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.026444 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:44.028985 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.028964 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 19:04:44.029095 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.028973 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:04:44.034859 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.034831 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds"] Jul 15 19:04:44.111746 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.111708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beeb75c5-4aa3-40dd-98ce-7391e4807762-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds\" (UID: \"beeb75c5-4aa3-40dd-98ce-7391e4807762\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:44.212310 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.212264 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beeb75c5-4aa3-40dd-98ce-7391e4807762-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds\" (UID: \"beeb75c5-4aa3-40dd-98ce-7391e4807762\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:44.213046 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.213023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beeb75c5-4aa3-40dd-98ce-7391e4807762-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds\" (UID: \"beeb75c5-4aa3-40dd-98ce-7391e4807762\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:44.338605 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.338576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:44.484149 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.484112 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds"] Jul 15 19:04:44.487015 ip-10-0-130-233 kubenswrapper[2557]: W0715 19:04:44.486986 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeeb75c5_4aa3_40dd_98ce_7391e4807762.slice/crio-ddb836596143e131497954a8e7449ecf991b808096d768f4e3a58477a78081d0 WatchSource:0}: Error finding container ddb836596143e131497954a8e7449ecf991b808096d768f4e3a58477a78081d0: Status 404 returned error can't find the container with id ddb836596143e131497954a8e7449ecf991b808096d768f4e3a58477a78081d0 Jul 15 19:04:44.967227 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.967133 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" event={"ID":"beeb75c5-4aa3-40dd-98ce-7391e4807762","Type":"ContainerStarted","Data":"3c8913b9539176cd9deab585165e3eed669bf006e01d012a455a1a0246796a11"} Jul 15 19:04:44.967227 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.967196 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" event={"ID":"beeb75c5-4aa3-40dd-98ce-7391e4807762","Type":"ContainerStarted","Data":"ddb836596143e131497954a8e7449ecf991b808096d768f4e3a58477a78081d0"} Jul 15 19:04:44.986224 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:44.986177 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" podStartSLOduration=0.986160046 podStartE2EDuration="986.160046ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:04:44.984104682 +0000 UTC m=+2865.593917870" watchObservedRunningTime="2026-07-15 19:04:44.986160046 +0000 UTC m=+2865.595973234" Jul 15 19:04:45.973247 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:45.973212 2557 generic.go:356] "Generic (PLEG): container finished" podID="beeb75c5-4aa3-40dd-98ce-7391e4807762" containerID="3c8913b9539176cd9deab585165e3eed669bf006e01d012a455a1a0246796a11" exitCode=0 Jul 15 19:04:45.973632 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:45.973274 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" event={"ID":"beeb75c5-4aa3-40dd-98ce-7391e4807762","Type":"ContainerDied","Data":"3c8913b9539176cd9deab585165e3eed669bf006e01d012a455a1a0246796a11"} Jul 15 19:04:47.112252 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.112224 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:47.138009 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.137975 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beeb75c5-4aa3-40dd-98ce-7391e4807762-maas-api-service-ca\") pod \"beeb75c5-4aa3-40dd-98ce-7391e4807762\" (UID: \"beeb75c5-4aa3-40dd-98ce-7391e4807762\") " Jul 15 19:04:47.138389 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.138364 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beeb75c5-4aa3-40dd-98ce-7391e4807762-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "beeb75c5-4aa3-40dd-98ce-7391e4807762" (UID: "beeb75c5-4aa3-40dd-98ce-7391e4807762"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:04:47.239602 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.239513 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beeb75c5-4aa3-40dd-98ce-7391e4807762-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:04:47.983864 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.983832 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" Jul 15 19:04:47.983864 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.983838 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds" event={"ID":"beeb75c5-4aa3-40dd-98ce-7391e4807762","Type":"ContainerDied","Data":"ddb836596143e131497954a8e7449ecf991b808096d768f4e3a58477a78081d0"} Jul 15 19:04:47.984080 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:47.983893 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddb836596143e131497954a8e7449ecf991b808096d768f4e3a58477a78081d0" Jul 15 19:04:54.524996 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:54.524896 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl"] Jul 15 19:04:54.527050 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:54.527029 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-401e9f-29735700-fc7xl"] Jul 15 19:04:55.997326 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:04:55.997293 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47029f8-bfa8-4eb0-b54b-4c29baebbedc" path="/var/lib/kubelet/pods/d47029f8-bfa8-4eb0-b54b-4c29baebbedc/volumes" Jul 15 19:05:09.855905 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:09.855871 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7"] Jul 15 19:05:09.856448 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:09.856130 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" containerID="cri-o://9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64" gracePeriod=30 Jul 15 19:05:10.273696 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.273612 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb"] Jul 15 19:05:10.274011 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.273998 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beeb75c5-4aa3-40dd-98ce-7391e4807762" containerName="revoke-keys" Jul 15 19:05:10.274058 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.274012 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="beeb75c5-4aa3-40dd-98ce-7391e4807762" containerName="revoke-keys" Jul 15 19:05:10.274094 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.274081 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="beeb75c5-4aa3-40dd-98ce-7391e4807762" containerName="revoke-keys" Jul 15 19:05:10.277100 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.277077 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:10.281649 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.281625 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2zdm\"" Jul 15 19:05:10.283493 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.283462 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:05:10.291477 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.291445 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb"] Jul 15 19:05:10.345101 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.345056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa495a7-2a87-4de5-974f-a56ff69846d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb\" (UID: \"0fa495a7-2a87-4de5-974f-a56ff69846d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:10.445906 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.445865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa495a7-2a87-4de5-974f-a56ff69846d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb\" (UID: \"0fa495a7-2a87-4de5-974f-a56ff69846d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:10.446551 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.446523 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa495a7-2a87-4de5-974f-a56ff69846d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb\" (UID: \"0fa495a7-2a87-4de5-974f-a56ff69846d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:10.594477 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.594443 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:10.746115 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:10.746091 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb"] Jul 15 19:05:10.748436 ip-10-0-130-233 kubenswrapper[2557]: W0715 19:05:10.748412 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa495a7_2a87_4de5_974f_a56ff69846d1.slice/crio-b578ec71a445f5dc4675c9c786ad31388715ca8f7cbf1391d1b1687ad171e336 WatchSource:0}: Error finding container b578ec71a445f5dc4675c9c786ad31388715ca8f7cbf1391d1b1687ad171e336: Status 404 returned error can't find the container with id b578ec71a445f5dc4675c9c786ad31388715ca8f7cbf1391d1b1687ad171e336 Jul 15 19:05:11.097957 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:11.097920 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" event={"ID":"0fa495a7-2a87-4de5-974f-a56ff69846d1","Type":"ContainerStarted","Data":"612836bdd0e1d0d4d35a3a644107d517332bb08feb0e8c6e96c62f67d3b0f530"} Jul 15 19:05:11.098346 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:11.097975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" event={"ID":"0fa495a7-2a87-4de5-974f-a56ff69846d1","Type":"ContainerStarted","Data":"b578ec71a445f5dc4675c9c786ad31388715ca8f7cbf1391d1b1687ad171e336"} Jul 15 19:05:11.119979 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:11.119930 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" podStartSLOduration=1.119914119 podStartE2EDuration="1.119914119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:05:11.118589302 +0000 UTC m=+2891.728402490" watchObservedRunningTime="2026-07-15 19:05:11.119914119 +0000 UTC m=+2891.729727301" Jul 15 19:05:12.104176 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:12.104140 2557 generic.go:356] "Generic (PLEG): container finished" podID="0fa495a7-2a87-4de5-974f-a56ff69846d1" containerID="612836bdd0e1d0d4d35a3a644107d517332bb08feb0e8c6e96c62f67d3b0f530" exitCode=0 Jul 15 19:05:12.104584 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:12.104221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" event={"ID":"0fa495a7-2a87-4de5-974f-a56ff69846d1","Type":"ContainerDied","Data":"612836bdd0e1d0d4d35a3a644107d517332bb08feb0e8c6e96c62f67d3b0f530"} Jul 15 19:05:13.256911 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:13.256879 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:13.374468 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:13.374369 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa495a7-2a87-4de5-974f-a56ff69846d1-maas-api-service-ca\") pod \"0fa495a7-2a87-4de5-974f-a56ff69846d1\" (UID: \"0fa495a7-2a87-4de5-974f-a56ff69846d1\") " Jul 15 19:05:13.374832 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:13.374807 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fa495a7-2a87-4de5-974f-a56ff69846d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0fa495a7-2a87-4de5-974f-a56ff69846d1" (UID: "0fa495a7-2a87-4de5-974f-a56ff69846d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:05:13.475859 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:13.475824 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fa495a7-2a87-4de5-974f-a56ff69846d1-maas-api-service-ca\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:14.115328 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:14.115291 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" event={"ID":"0fa495a7-2a87-4de5-974f-a56ff69846d1","Type":"ContainerDied","Data":"b578ec71a445f5dc4675c9c786ad31388715ca8f7cbf1391d1b1687ad171e336"} Jul 15 19:05:14.115328 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:14.115325 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb" Jul 15 19:05:14.115568 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:14.115352 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b578ec71a445f5dc4675c9c786ad31388715ca8f7cbf1391d1b1687ad171e336" Jul 15 19:05:14.400885 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:14.400849 2557 patch_prober.go:28] interesting pod/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 19:05:14.401263 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:14.400918 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 19:05:15.109591 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.109564 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 19:05:15.121869 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.121824 2557 generic.go:356] "Generic (PLEG): container finished" podID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerID="9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64" exitCode=0 Jul 15 19:05:15.121869 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.121870 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" Jul 15 19:05:15.122132 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.121906 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" event={"ID":"9f8921bc-17b2-4392-b811-f745a0ef7ebd","Type":"ContainerDied","Data":"9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64"} Jul 15 19:05:15.122132 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.121961 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7" event={"ID":"9f8921bc-17b2-4392-b811-f745a0ef7ebd","Type":"ContainerDied","Data":"d261d32251d9d6352a4331ebae8194384f4143910903f5740a5309aa5336cbb6"} Jul 15 19:05:15.122132 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.121987 2557 scope.go:117] "RemoveContainer" containerID="9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64" Jul 15 19:05:15.131221 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.131197 2557 scope.go:117] "RemoveContainer" containerID="230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3" Jul 15 19:05:15.141635 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.141613 2557 scope.go:117] "RemoveContainer" containerID="9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64" Jul 15 19:05:15.142013 ip-10-0-130-233 kubenswrapper[2557]: E0715 19:05:15.141986 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64\": container with ID starting with 9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64 not found: ID does not exist" containerID="9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64" Jul 15 19:05:15.142087 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.142030 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64"} err="failed to get container status \"9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64\": rpc error: code = NotFound desc = could not find container \"9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64\": container with ID starting with 9720f04c3492ba35e92b41a4035b28cf6d61eda5a1c1c742dc4a9a55dc78bf64 not found: ID does not exist" Jul 15 19:05:15.142087 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.142056 2557 scope.go:117] "RemoveContainer" containerID="230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3" Jul 15 19:05:15.142301 ip-10-0-130-233 kubenswrapper[2557]: E0715 19:05:15.142282 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3\": container with ID starting with 230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3 not found: ID does not exist" containerID="230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3" Jul 15 19:05:15.142344 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.142312 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3"} err="failed to get container status \"230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3\": rpc error: code = NotFound desc = could not find container \"230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3\": container with ID starting with 230e55a6ec7fcf1c7a7938d09c5fd00b4bd71b32658aef749a842ed7cba6bba3 not found: ID does not exist" Jul 15 19:05:15.190736 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190694 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-podinfo\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.190736 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190735 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-credential-socket\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.190997 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190830 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-data\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.190997 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190854 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istiod-ca-cert\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.190997 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190890 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-token\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.190997 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190944 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbftj\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-kube-api-access-vbftj\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.190997 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.190975 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-certs\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.191254 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191005 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-envoy\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.191254 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191035 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-socket\") pod \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\" (UID: \"9f8921bc-17b2-4392-b811-f745a0ef7ebd\") " Jul 15 19:05:15.191254 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191169 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:15.191405 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191308 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-credential-socket\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.191405 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191353 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:05:15.191527 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191496 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:15.191589 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.191569 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:15.192153 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.192125 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:15.193080 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.193056 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 19:05:15.193277 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.193247 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-kube-api-access-vbftj" (OuterVolumeSpecName: "kube-api-access-vbftj") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "kube-api-access-vbftj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:05:15.193336 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.193255 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:05:15.193336 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.193258 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f8921bc-17b2-4392-b811-f745a0ef7ebd" (UID: "9f8921bc-17b2-4392-b811-f745a0ef7ebd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:05:15.292217 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292183 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbftj\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-kube-api-access-vbftj\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292217 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292213 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-certs\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292217 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292223 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-envoy\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292454 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292232 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-workload-socket\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292454 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292242 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-podinfo\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292454 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292251 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-data\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292454 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292259 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istiod-ca-cert\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.292454 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.292266 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8921bc-17b2-4392-b811-f745a0ef7ebd-istio-token\") on node \"ip-10-0-130-233.ec2.internal\" DevicePath \"\"" Jul 15 19:05:15.445730 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.445697 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7"] Jul 15 19:05:15.451664 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:15.451636 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-401e9f-openshift-default-596466fff9-xc5w7"] Jul 15 19:05:16.001738 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:16.001705 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" path="/var/lib/kubelet/pods/9f8921bc-17b2-4392-b811-f745a0ef7ebd/volumes" Jul 15 19:05:20.827450 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:20.827407 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2"] Jul 15 19:05:20.835094 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:20.835062 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-46e17a-29735700-59bd2"] Jul 15 19:05:21.997276 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:21.997242 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4effdf43-d7a5-4dd1-bd3f-4be06aadf131" path="/var/lib/kubelet/pods/4effdf43-d7a5-4dd1-bd3f-4be06aadf131/volumes" Jul 15 19:05:42.960845 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:42.960807 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g6bh4_c97fb344-f922-4fa9-adb5-4f6a5498673d/manager/0.log" Jul 15 19:05:43.332709 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:43.332608 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5664889d69-wsdn5_e4b7adc6-3fd2-4930-b357-2ca42676d8ce/copy-manifests/0.log" Jul 15 19:05:43.340519 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:43.340496 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5664889d69-wsdn5_e4b7adc6-3fd2-4930-b357-2ca42676d8ce/manager/0.log" Jul 15 19:05:43.941185 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:43.941150 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735685-fcp5d_dedb3d60-8510-46ef-8283-4debd86915dd/cleanup/0.log" Jul 15 19:05:44.178811 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.178782 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-3f3fe1b2-lxxs6_0184b5bd-c193-4543-99d8-7c22fcd36998/revoke-keys/0.log" Jul 15 19:05:44.294902 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.294819 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-cfe32069-2lzlw_412badb1-0609-4de3-82c6-423d55cfe749/revoke-keys/0.log" Jul 15 19:05:44.408905 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.408880 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-a0cde7-btkt6_7ca7fef1-15cd-4f03-a9ea-21562a1c3e62/revoke-keys/0.log" Jul 15 19:05:44.529434 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.529410 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-dc683a-2tvz2_d86cff8b-9b6a-484d-879b-49e5d5c614a2/revoke-keys/0.log" Jul 15 19:05:44.647417 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.647383 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-8020c7cf-v4n2r_4d257934-f5cb-4f47-92f7-d29c2d9832f1/revoke-keys/0.log" Jul 15 19:05:44.761928 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.761893 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-cbf10e-b874t_d8f1c12b-b3ee-4c2b-8ba3-e5aafc80953d/revoke-keys/0.log" Jul 15 19:05:44.883343 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:44.883312 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-f83daa-cfh5v_192d3916-6f36-4788-b922-327555b03ab5/revoke-keys/0.log" Jul 15 19:05:45.002461 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:45.002386 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-827ccb59-nrgj6_c38348d3-7fbf-49de-ac17-c1e7e76c9f8c/revoke-keys/0.log" Jul 15 19:05:45.121548 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:45.121517 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-24b88ef5-nqkms_d06ea689-ffa4-44c4-93c4-c2db0bbc5724/revoke-keys/0.log" Jul 15 19:05:45.247314 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:45.247289 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-ba7662e4-c42n2_3f533c1f-6d62-407b-bb7a-9afe233b1e9a/revoke-keys/0.log" Jul 15 19:05:45.364272 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:45.364246 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-401e9f-t77ds_beeb75c5-4aa3-40dd-98ce-7391e4807762/revoke-keys/0.log" Jul 15 19:05:45.475324 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:45.475290 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-46e17a-dqmfb_0fa495a7-2a87-4de5-974f-a56ff69846d1/revoke-keys/0.log" Jul 15 19:05:48.225249 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:48.225200 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786slhj_b4db6401-7924-468b-98a8-16288d2304ee/istio-proxy/0.log" Jul 15 19:05:48.363760 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:48.363720 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mjqlw_987f8e4b-7f84-4201-bff9-1475aa8dc4c3/discovery/0.log" Jul 15 19:05:48.868284 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:48.868250 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mtl2j_7eb63288-7bb5-45c4-8e1f-4a416d1d5306/payload-pre-processing/0.log" Jul 15 19:05:49.708212 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:49.708176 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8x4nv_f1c44ea9-96f7-4f70-899c-cd45ef545d69/main/0.log" Jul 15 19:05:49.828699 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:49.828672 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db84pqm_0c1f34f8-eceb-4216-9e2c-5583332bdea1/main/0.log" Jul 15 19:05:59.456841 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:59.456805 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2n6fz_19672c70-677e-46a2-9a56-399fc656d1a5/global-pull-secret-syncer/0.log" Jul 15 19:05:59.695348 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:59.695314 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tvz6c_1bb98a3f-e75a-4b4a-a92f-9bb9feed6542/konnectivity-agent/0.log" Jul 15 19:05:59.726103 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:05:59.725997 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-233.ec2.internal_27d6e657e0fc42c84018a1763a94d169/haproxy/0.log" Jul 15 19:06:07.415004 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:07.414952 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f8cpq_28d1c725-9a43-42f5-9439-43ddb744be24/node-exporter/0.log" Jul 15 19:06:07.438115 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:07.438077 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f8cpq_28d1c725-9a43-42f5-9439-43ddb744be24/kube-rbac-proxy/0.log" Jul 15 19:06:07.459363 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:07.459334 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f8cpq_28d1c725-9a43-42f5-9439-43ddb744be24/init-textfile/0.log" Jul 15 19:06:09.437981 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.437949 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-tzkcr_305b6e0c-0c28-4d1e-bacf-09ae4e2af7d9/networking-console-plugin/0.log" Jul 15 19:06:09.827296 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827214 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck"] Jul 15 19:06:09.827573 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827559 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" Jul 15 19:06:09.827614 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827575 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" Jul 15 19:06:09.827614 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827589 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fa495a7-2a87-4de5-974f-a56ff69846d1" containerName="revoke-keys" Jul 15 19:06:09.827614 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827594 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fa495a7-2a87-4de5-974f-a56ff69846d1" containerName="revoke-keys" Jul 15 19:06:09.827614 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827610 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" Jul 15 19:06:09.827740 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827619 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" Jul 15 19:06:09.827740 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827674 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8921bc-17b2-4392-b811-f745a0ef7ebd" containerName="istio-proxy" Jul 15 19:06:09.827740 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.827685 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fa495a7-2a87-4de5-974f-a56ff69846d1" containerName="revoke-keys" Jul 15 19:06:09.830754 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.830732 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:09.833225 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.833199 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-858dh\"/\"openshift-service-ca.crt\"" Jul 15 19:06:09.834158 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.834136 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-858dh\"/\"kube-root-ca.crt\"" Jul 15 19:06:09.834257 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.834212 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-858dh\"/\"default-dockercfg-td7m4\"" Jul 15 19:06:09.843795 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.843751 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck"] Jul 15 19:06:09.972963 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.972925 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-podres\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:09.972963 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.972966 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-lib-modules\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:09.973185 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.973072 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlhmb\" (UniqueName: \"kubernetes.io/projected/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-kube-api-access-tlhmb\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:09.973185 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.973137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-sys\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:09.973185 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:09.973177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-proc\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074604 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlhmb\" (UniqueName: \"kubernetes.io/projected/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-kube-api-access-tlhmb\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074865 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-sys\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074865 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-proc\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074865 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074718 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-podres\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074865 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074744 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-lib-modules\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074865 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074792 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-sys\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.074865 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074801 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-proc\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.075092 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074907 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-lib-modules\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.075092 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.074915 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-podres\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.087433 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.087406 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlhmb\" (UniqueName: \"kubernetes.io/projected/bc913d9f-2e5a-4860-a972-d6928d1f3cd9-kube-api-access-tlhmb\") pod \"perf-node-gather-daemonset-mlfck\" (UID: \"bc913d9f-2e5a-4860-a972-d6928d1f3cd9\") " pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.090023 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.089994 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/1.log" Jul 15 19:06:10.098802 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.098759 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kplvm_a1707097-c6df-4856-b7aa-426736ef40ff/console-operator/2.log" Jul 15 19:06:10.140847 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.140811 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.278670 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.278644 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck"] Jul 15 19:06:10.280895 ip-10-0-130-233 kubenswrapper[2557]: W0715 19:06:10.280867 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbc913d9f_2e5a_4860_a972_d6928d1f3cd9.slice/crio-a23f21b04af75944f6474818ff01262d8431924feeecbcef0d0e6e85fa058a1f WatchSource:0}: Error finding container a23f21b04af75944f6474818ff01262d8431924feeecbcef0d0e6e85fa058a1f: Status 404 returned error can't find the container with id a23f21b04af75944f6474818ff01262d8431924feeecbcef0d0e6e85fa058a1f Jul 15 19:06:10.415496 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.415463 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" event={"ID":"bc913d9f-2e5a-4860-a972-d6928d1f3cd9","Type":"ContainerStarted","Data":"de54104f3ba76b29406c7d2dc40534f6f5a394894fd657d3d0feda4755ee487a"} Jul 15 19:06:10.415678 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.415522 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" event={"ID":"bc913d9f-2e5a-4860-a972-d6928d1f3cd9","Type":"ContainerStarted","Data":"a23f21b04af75944f6474818ff01262d8431924feeecbcef0d0e6e85fa058a1f"} Jul 15 19:06:10.415678 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.415545 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:10.433908 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:10.433853 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" podStartSLOduration=1.433835564 podStartE2EDuration="1.433835564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:06:10.432398508 +0000 UTC m=+2951.042211697" watchObservedRunningTime="2026-07-15 19:06:10.433835564 +0000 UTC m=+2951.043648764" Jul 15 19:06:11.215180 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:11.215151 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-gdzmt_a527b261-efe0-492b-9523-6291d637a90d/volume-data-source-validator/0.log" Jul 15 19:06:12.317286 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:12.317259 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xx5wd_34cf3357-ec56-4e44-8b1a-58379395479b/dns/0.log" Jul 15 19:06:12.343187 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:12.343156 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xx5wd_34cf3357-ec56-4e44-8b1a-58379395479b/kube-rbac-proxy/0.log" Jul 15 19:06:12.367210 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:12.367177 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-56pv4_f6ca71a2-f8c6-4eb3-8f5e-c5d0e12a9a3e/dns-node-resolver/0.log" Jul 15 19:06:12.951368 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:12.951335 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64c86bbcb8-d6ggr_c8bba59a-4efb-41f8-817b-8a04c1bee4b6/registry/0.log" Jul 15 19:06:13.027006 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:13.026977 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xd77m_19430ccd-cec5-413d-a155-58a1049530be/node-ca/0.log" Jul 15 19:06:13.973581 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:13.973553 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786slhj_b4db6401-7924-468b-98a8-16288d2304ee/istio-proxy/0.log" Jul 15 19:06:14.188710 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:14.188673 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mjqlw_987f8e4b-7f84-4201-bff9-1475aa8dc4c3/discovery/0.log" Jul 15 19:06:14.349625 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:14.349583 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mtl2j_7eb63288-7bb5-45c4-8e1f-4a416d1d5306/payload-pre-processing/0.log" Jul 15 19:06:15.101257 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:15.101223 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-htk6w_550040cc-16c3-4d74-80a0-908ee435260a/serve-healthcheck-canary/0.log" Jul 15 19:06:15.811581 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:15.811553 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qnm4m_38192496-f9a3-42ff-88db-26a2d9452e2a/kube-rbac-proxy/0.log" Jul 15 19:06:15.834796 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:15.834746 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qnm4m_38192496-f9a3-42ff-88db-26a2d9452e2a/exporter/0.log" Jul 15 19:06:15.859743 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:15.859700 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qnm4m_38192496-f9a3-42ff-88db-26a2d9452e2a/extractor/0.log" Jul 15 19:06:16.430466 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:16.430442 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-858dh/perf-node-gather-daemonset-mlfck" Jul 15 19:06:18.426978 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:18.426940 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g6bh4_c97fb344-f922-4fa9-adb5-4f6a5498673d/manager/0.log" Jul 15 19:06:18.628081 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:18.628040 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5664889d69-wsdn5_e4b7adc6-3fd2-4930-b357-2ca42676d8ce/manager/0.log" Jul 15 19:06:18.649544 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:18.649518 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5664889d69-wsdn5_e4b7adc6-3fd2-4930-b357-2ca42676d8ce/copy-manifests/0.log" Jul 15 19:06:19.309734 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:19.309705 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786slhj_b4db6401-7924-468b-98a8-16288d2304ee/istio-proxy/0.log" Jul 15 19:06:19.518121 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:19.518087 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mjqlw_987f8e4b-7f84-4201-bff9-1475aa8dc4c3/discovery/0.log" Jul 15 19:06:19.640708 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:19.640677 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mtl2j_7eb63288-7bb5-45c4-8e1f-4a416d1d5306/payload-pre-processing/0.log" Jul 15 19:06:19.961553 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:19.961526 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68b4879c65-r8zwz_c43dbd64-9f18-433e-a88b-d3b491c0eea2/manager/0.log" Jul 15 19:06:21.082845 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:21.082816 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786slhj_b4db6401-7924-468b-98a8-16288d2304ee/istio-proxy/0.log" Jul 15 19:06:21.301752 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:21.301673 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-mjqlw_987f8e4b-7f84-4201-bff9-1475aa8dc4c3/discovery/0.log" Jul 15 19:06:21.429890 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:21.429864 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-mtl2j_7eb63288-7bb5-45c4-8e1f-4a416d1d5306/payload-pre-processing/0.log" Jul 15 19:06:25.158787 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:25.158738 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-mzhxb_ba8f959a-ecc5-466b-b411-2198a71967cd/migrator/0.log" Jul 15 19:06:25.182325 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:25.182291 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-mzhxb_ba8f959a-ecc5-466b-b411-2198a71967cd/graceful-termination/0.log" Jul 15 19:06:25.577379 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:25.577288 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-4pthj_ea3d3efe-2f9f-4b75-9521-e95e7d1b624a/kube-storage-version-migrator-operator/1.log" Jul 15 19:06:25.579097 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:25.579073 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-4pthj_ea3d3efe-2f9f-4b75-9521-e95e7d1b624a/kube-storage-version-migrator-operator/0.log" Jul 15 19:06:26.589436 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.589401 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-96qbj_e877bfe5-3035-4a48-b6d5-7c321c02fcf7/kube-multus/0.log" Jul 15 19:06:26.617262 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.617235 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/kube-multus-additional-cni-plugins/0.log" Jul 15 19:06:26.639882 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.639856 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/egress-router-binary-copy/0.log" Jul 15 19:06:26.662635 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.662606 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/cni-plugins/0.log" Jul 15 19:06:26.688850 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.688820 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/bond-cni-plugin/0.log" Jul 15 19:06:26.712678 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.712653 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/routeoverride-cni/0.log" Jul 15 19:06:26.735716 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.735684 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/whereabouts-cni-bincopy/0.log" Jul 15 19:06:26.760073 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:26.760041 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-82tdr_5bdcafe2-1ea6-4997-bf00-8471961ef963/whereabouts-cni/0.log" Jul 15 19:06:27.267317 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:27.267272 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-csskv_97944bdc-1dea-4012-ac18-4827adcd2da6/network-metrics-daemon/0.log" Jul 15 19:06:27.289641 ip-10-0-130-233 kubenswrapper[2557]: I0715 19:06:27.289598 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-csskv_97944bdc-1dea-4012-ac18-4827adcd2da6/kube-rbac-proxy/0.log"